[](./examples/Youtube-Search-QA-Bot/main.py) [](./examples/Youtube-Search-QA-Bot/index.js)||
-| [Langchain: Code Docs QA bot](/examples/Code-Documentation-QA-Bot/) | [](./examples/Code-Documentation-QA-Bot/main.py)[](./examples/Code-Documentation-QA-Bot/index.js)||
-| [AI Agents: Reducing Hallucination](/examples/reducing_hallucinations_ai_agents/) | [](./examples/reducing_hallucinations_ai_agents/main.py) [](./examples/reducing_hallucinations_ai_agents/index.js)| [](https://blog.lancedb.com/how-to-reduce-hallucinations-from-llm-powered-agents-using-long-term-memory-72f262c3cc1f)|
-| [Multimodal CLIP: DiffusionDB](/examples/multimodal_clip/) | [](./examples/multimodal_clip/main.py)| [](https://medium.com/etoai/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939)|
-| [Multimodal CLIP: Youtube videos](/examples/multimodal_video_search/) | [](./examples/multimodal_video_search/main.py) |[](https://medium.com/etoai/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939)|
-| [Multimodal Image + Text Search](/examples/multimodal_search/) | [](./examples/multimodal_search/main.py)| [](https://medium.com/etoai/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939) |
-| [TransformersJS Embedding example](./examples/js-transformers/) |[](./examples/js-transformers/index.js) | |
-| [Movie Recommender](/examples/movie-recommender/) | [](./examples/movie-recommender/main.py)| |
+| [Youtube transcript search bot](/examples/Youtube-Search-QA-Bot/) | [](./examples/Youtube-Search-QA-Bot/main.py) [](./examples/Youtube-Search-QA-Bot/index.js) [](#)||
+| [Langchain: Code Docs QA bot](/examples/Code-Documentation-QA-Bot/) | [](./examples/Code-Documentation-QA-Bot/main.py) [](./examples/Code-Documentation-QA-Bot/index.js)[](#)||
+| [AI Agents: Reducing Hallucination](/examples/reducing_hallucinations_ai_agents/) | [](./examples/reducing_hallucinations_ai_agents/main.py) [](./examples/reducing_hallucinations_ai_agents/index.js)[](#)| [](https://blog.lancedb.com/how-to-reduce-hallucinations-from-llm-powered-agents-using-long-term-memory-72f262c3cc1f)|
+| [Multimodal CLIP: DiffusionDB](/examples/multimodal_clip_diffusiondb/) | [](./examples/multimodal_clip_diffusiondb/main.py) [](#)| [](https://medium.com/etoai/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939)|
+| [Multimodal CLIP: Youtube videos](/examples/multimodal_video_search/) | [](./examples/multimodal_video_search/main.py) [](#) |[](https://medium.com/etoai/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939)|
+| [Multimodal Image + Text Search](/examples/multimodal_search/) | [](./examples/multimodal_search/main.py) [](#)| [](https://medium.com/etoai/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939) |
+| [TransformersJS Embedding example](./examples/js-transformers/) |[](./examples/js-transformers/index.js) [](#) | |
+| [Movie Recommender](/examples/movie-recommender/) | [](./examples/movie-recommender/main.py)| |
| [Product Recommender](./examples/product-recommender/) | [](./examples/product-recommender/main.py) | |
-| [Audio Search](./examples/audio_search/) | [](./examples/audio_search/main.py) | |
-| [Arxiv paper recommender](/examples/arxiv-recommender) | [](./examples/arxiv-recommender/main.py) | |
-| [Multi-lingual search](/examples/multi-lingual-wiki-qa) | [](.examples/multi-lingual-wiki-qa/main.py) | |
-| [Instruct-Multitask](./examples/instruct-multitask) | [](.examples/instruct-multitask/main.py) | [](https://blog.lancedb.com/multitask-embedding-with-lancedb-be18ec397543)|
-| [Improve RAG with Re-ranking](/examples/RAG_re_ranking/) | |[](https://blog.lancedb.com/simplest-method-to-improve-rag-pipeline-re-ranking-cf6eaec6d544)|
-| [Improve RAG with FLARE](/examples/Advanced-RAG-with-FLARE) | [](.examples/instruct-multitask/main.py) | [](https://blog.lancedb.com/better-rag-with-active-retrieval-augmented-generation-flare-3b66646e2a9f)|
-| [Improve RAG with HyDE](/examples/Advance-RAG-with-HyDE/) | |[](https://blog.lancedb.com/advanced-rag-precise-zero-shot-dense-retrieval-with-hyde-0946c54dfdcb)|
-| [Improve RAG with LOTR ](/examples/Advance_RAG_LOTR/) | |[](https://blog.lancedb.com/better-rag-with-lotr-lord-of-retriever-23c8336b9a35)|
-| [Advanced RAG: Parent Document Retriever](/examples/parent_document_retriever/) | |[](https://blog.lancedb.com/modified-rag-parent-document-bigger-chunk-retriever-62b3d1e79bc6)|
-| [RAG Fusion](/examples/RAG_Fusion/) | |
-| [Hybrid search BM25 & lancedb ](./examples/Hybrid_search_bm25_lancedb/) | |[](https://blog.lancedb.com/hybrid-search-combining-bm25-and-semantic-search-for-better-results-with-lan-1358038fe7e6)|
-| [Evaluating Prompts with Prompttools](/examples/prompttools-eval-prompts/) | | |
-| [NER powered with Semantic Search](/tutorials/NER-powered-Semantic-Search/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/NER-powered-Semantic-Search/NER_powered_Semantic_Search_with_LanceDB.ipynb) | [](https://blog.lancedb.com/ner-powered-semantic-search-using-lancedb-51051dc3e493)|
-[Sentiment Analysis : Analysing Hotel Reviews](/examples/Sentiment-Analysis-Analyse-Hotel-Reviews/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Sentiment-Analysis-Analyse-Hotel-Reviews/Sentiment_Analysis_using_LanceDB.ipynb) | [](https://blog.lancedb.com/sentiment-analysis-using-lancedb-2da3cb1e3fa6)|
+| [Audio Search](./examples/audio_search/) | [](./examples/audio_search/main.py) [](#) | |
+| [Arxiv paper recommender](/examples/arxiv-recommender) | [](./examples/arxiv-recommender/main.py) [](#) | |
+| [Multi-lingual search](/examples/multi-lingual-wiki-qa) | [](.examples/multi-lingual-wiki-qa/main.py) [](#) | |
+| [Instruct-Multitask](./examples/instruct-multitask) | [](.examples/instruct-multitask/main.py) [](#) | [](https://blog.lancedb.com/multitask-embedding-with-lancedb-be18ec397543)|
+| [Improve RAG with Re-ranking](/examples/RAG_re_ranking/) | [](#) |[](https://blog.lancedb.com/simplest-method-to-improve-rag-pipeline-re-ranking-cf6eaec6d544)|
+| [Improve RAG with FLARE](/examples/Advanced-RAG-with-FLARE) | [](.examples/instruct-multitask/main.py) [](#) | [](https://blog.lancedb.com/better-rag-with-active-retrieval-augmented-generation-flare-3b66646e2a9f)|
+| [Improve RAG with HyDE](/examples/Advance-RAG-with-HyDE/) | [](#)|[](https://blog.lancedb.com/advanced-rag-precise-zero-shot-dense-retrieval-with-hyde-0946c54dfdcb)|
+| [Improve RAG with LOTR ](/examples/Advance_RAG_LOTR/) | [](#) |[](https://blog.lancedb.com/better-rag-with-lotr-lord-of-retriever-23c8336b9a35)|
+| [Advanced RAG: Parent Document Retriever](/examples/parent_document_retriever/) | [](#)|[](https://blog.lancedb.com/modified-rag-parent-document-bigger-chunk-retriever-62b3d1e79bc6)|
+| [RAG Fusion](/examples/RAG_Fusion/) | [](#)|
+| [Hybrid search BM25 & lancedb ](./examples/Hybrid_search_bm25_lancedb/) | [](#)|[](https://blog.lancedb.com/hybrid-search-combining-bm25-and-semantic-search-for-better-results-with-lan-1358038fe7e6)|
+| [Evaluating Prompts with Prompttools](/examples/prompttools-eval-prompts/) | [](#) [](#)| |
+| [NER powered with Semantic Search](/tutorials/NER-powered-Semantic-Search/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/NER-powered-Semantic-Search/NER_powered_Semantic_Search_with_LanceDB.ipynb)[](#) | [](https://blog.lancedb.com/ner-powered-semantic-search-using-lancedb-51051dc3e493)|
+[Sentiment Analysis : Analysing Hotel Reviews](/examples/Sentiment-Analysis-Analyse-Hotel-Reviews/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Sentiment-Analysis-Analyse-Hotel-Reviews/Sentiment_Analysis_using_LanceDB.ipynb) [](#) | [](https://blog.lancedb.com/sentiment-analysis-using-lancedb-2da3cb1e3fa6)|
| [Facial Recognition](./examples/facial_recognition) | |
-| [Accelerate Vector Search Applications Using OpenVINO](/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/clip_text_image_search.ipynb) | [](https://blog.lancedb.com/accelerate-vector-search-applications-using-openvino-51366eabf866)|
-| [Search Within Images](/examples/Contextual-Compression-with-RAG/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/search-within-images-with-sam-and-clip/main.ipynb) |[](https://blog.lancedb.com/search-within-an-image-331b54e4285e)|
-| [Contextual-Compression-with-RAG](/examples/Contextual-Compression-with-RAG/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Contextual-Compression-with-RAG/main.ipynb) |[](https://medium.com/etoai/enhance-rag-integrate-contextual-compression-and-filtering-for-precision-a29d4a810301)
+| [Accelerate Vector Search Applications Using OpenVINO](/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/clip_text_image_search.ipynb) [](#)| [](https://blog.lancedb.com/accelerate-vector-search-applications-using-openvino-51366eabf866)|
+| [Search Within Images](/examples/Contextual-Compression-with-RAG/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/search-within-images-with-sam-and-clip/main.ipynb) [](#) |[](https://blog.lancedb.com/search-within-an-image-331b54e4285e)|
+| [Contextual-Compression-with-RAG](/examples/Contextual-Compression-with-RAG/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Contextual-Compression-with-RAG/main.ipynb) [](#) |[](https://medium.com/etoai/enhance-rag-integrate-contextual-compression-and-filtering-for-precision-a29d4a810301)
@@ -73,20 +73,21 @@ These are ready to use applications built using LanceDB serverless vector databa
| [Multi-Modal Search Engine](https://github.com/lancedb/vectordb-recipes/tree/rf/applications/multimodal-search) | Create a Multi-modal search engine app, to search images using both images or text | |
| [ Chat with multiple URL/website ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/chat_with_anywebsite/) | Conversational AI for Any Website with Mistral,Bge Embedding & LanceDB | |
| [ Hr chatbot ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/HR_chatbot/) | Hr chatbot - ask your personal query using zero-shot React agent & tools ||
-| [ Talk with Youtube Video using GPT4 Vision API ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/talk-with-youtube-gpt4-vision-api/) | Talk with Youtube Video using GPT4 Vision API and Langchain | |
-| [ Talk with Podcast ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/talk-with-podcast) | Talk with Youtube Podcast using Ollama and insanely-fast-whisper | |
-| [ Talk with Wikipedia ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/talk-with-wikipedia) | Talk with Wikipedia Pages | |
+| [ Talk with Youtube Video using GPT4 Vision API ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/talk-with-youtube-gpt4-vision-api/) | Talk with Youtube Video using GPT4 Vision API and Langchain | |
+| [ Talk with Podcast ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/talk-with-podcast) | Talk with Youtube Podcast using Ollama and insanely-fast-whisper | |
+| [ Talk with Wikipedia ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/talk-with-wikipedia) | Talk with Wikipedia Pages | |
+| [ Talk with Github ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/talk-with-github) | Talk with Github Codespaces using Qwen1.5 | |
| [ Document Chat with Langroid ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/docchat-with-langroid) | Talk with your Documents using Langroid | |
| [ Fastapi RAG template ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/Chatbot_RAG_with_FASTAPI) | FastAPI based RAG template with Websocket support | |
| [ GTE MLX RAG ](https://github.com/lancedb/vectordb-recipes/tree/main/applications/GTE_mlx_RAG/CLI_example.ipynb) | mlx based RAG model using lancedb api support | |
-
-
+| [Multilingual-RAG](https://github.com/lancedb/vectordb-recipes/tree/main/applications/Multilingual_RAG/) | Multilingual RAG with cohere embedding & support 100+ languages||
## Tutorials
Looking to get started with LLMs, vectorDBs, and the world of Generative AI? These in-depth tutorials and courses cover these concepts with practical follow along colabs where possible.
| Tutorial | Interactive Environment | Blog Link |
| --------- | -------------------------- | ----------- |
| | | |
+| [Corrective RAG with Langgraph](./tutorials/Corrective-RAG-with_Langgraph/) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/Corrective-RAG-with_Langgraph/CRAG_with_Langgraph.ipynb) | |
| [Product Quantization: Compress High Dimensional Vectors](https://blog.lancedb.com/product-quantization-compress-high-dimensional-vectors-dfcba98fab47) | | [](https://blog.lancedb.com/product-quantization-compress-high-dimensional-vectors-dfcba98fab47) |
| [LLMs, RAG, & the missing storage layer for AI](https://medium.com/etoai/llms-rag-the-missing-storage-layer-for-ai-28ded35fa984) | | [](https://medium.com/etoai/llms-rag-the-missing-storage-layer-for-ai-28ded35fa984) |
| [Fine-Tuning LLM using PEFT & QLoRA](./tutorials/fine-tuning_LLM_with_PEFT_QLoRA) | [](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/fine-tuning_LLM_with_PEFT_QLoRA/main.ipynb) | [](https://blog.lancedb.com/optimizing-llms-a-step-by-step-guide-to-fine-tuning-with-peft-and-qlora-22eddd13d25b) |
diff --git a/applications/Multilingual_RAG/.env-example b/applications/Multilingual_RAG/.env-example
new file mode 100644
index 0000000..39553fb
--- /dev/null
+++ b/applications/Multilingual_RAG/.env-example
@@ -0,0 +1 @@
+COHERE_API_KEY = pastyourapikeyhere
diff --git a/applications/Multilingual_RAG/README.md b/applications/Multilingual_RAG/README.md
new file mode 100644
index 0000000..7f77340
--- /dev/null
+++ b/applications/Multilingual_RAG/README.md
@@ -0,0 +1,45 @@
+# Multilingual-RAG
+
+
+
+## Overview
+Multilingual-RAG is an innovative question-answering system with multilingual capabilities, capable of understanding and generating responses in multiple languages. It is built upon the powerful architecture of Large Language Models (LLMs) with Retrieve-And-Generate (RAG) capabilities. This application harnesses the capabilities of Cohere's multilingual embeddings, LanceDB vector store, LangChain for question answering, and Argos Translate for seamless translation between languages. The user interface is provided by Gradio, ensuring a smooth and interactive user experience.
+
+## Supported Languages
+Multilingual RAG is designed to support over 100 languages. The specific list of supported languages depends on the capabilities of the Cohere multilingual model and Argos Translate. By default, it includes support for English, Hindi, French, and Turkish languages. Additional languages can be added to suit your use case.
+
+## Getting Started
+Follow these instructions to set up Multilingual-RAG in your local environment.
+
+### Prerequisites
+Ensure you have the following prerequisites installed:
+- Python 3.x
+
+Create a `.env` file and add your Cohere API key:
+just rename `.env-example` with `.env` & past your API
+
+
+
+## Installation
+You can install the required dependencies using the following commands:
+
+```
+pip install -r requirements.txt
+```
+For Argos Translate, you can install it as follows:
+
+```
+git clone https://github.com/argosopentech/argos-translate.git
+cd argos-translate
+virtualenv env
+source env/bin/activate
+pip install -e .
+```
+
+## Running the App
+To run the Multilingual-RAG app, use the following command:
+Currently, support text/pdf file - change the file path inside main.py
+
+```
+python3 main.py
+```
diff --git a/applications/Multilingual_RAG/main.py b/applications/Multilingual_RAG/main.py
new file mode 100644
index 0000000..269d33b
--- /dev/null
+++ b/applications/Multilingual_RAG/main.py
@@ -0,0 +1,218 @@
+import os
+import dotenv
+import gradio as gr
+import lancedb
+import logging
+from langchain.embeddings.cohere import CohereEmbeddings
+from langchain.llms import Cohere
+from langchain.prompts import PromptTemplate
+from langchain.chains import RetrievalQA
+from langchain.vectorstores import LanceDB
+from langchain.document_loaders import TextLoader
+from langchain.text_splitter import RecursiveCharacterTextSplitter
+from langchain_community.document_loaders import PyPDFLoader
+import argostranslate.package
+import argostranslate.translate
+
+
+# Configuration Management
+dotenv.load_dotenv(".env")
+DB_PATH = "/tmp/lancedb"
+
+COHERE_MODEL_NAME = "multilingual-22-12"
+LANGUAGE_ISO_CODES = {
+ "English": "en",
+ "Hindi": "hi",
+ "Turkish": "tr",
+ "French": "fr",
+}
+
+# Logging Configuration
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+def initialize_documents_and_embeddings(input_file_path):
+ """
+ Initialize documents and their embeddings from a given file.
+
+ Parameters:
+ - input_file_path (str): The path to the input file. Supported formats are .txt and .pdf.
+
+ Returns:
+ - tuple: A tuple containing a list of texts split from the document and the embeddings object.
+ """
+ file_extension = os.path.splitext(input_file_path)[1]
+ if file_extension == ".txt":
+ logger.info("txt file processing")
+ # Handle text file
+ loader = TextLoader(input_file_path)
+ documents = loader.load()
+ text_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=50)
+ texts = text_splitter.split_documents(documents)
+ elif file_extension == ".pdf":
+ logger.info("pdf file processing")
+ # Handle PDF file
+ loader = PyPDFLoader(input_file_path)
+ texts = loader.load_and_split()
+ text_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=50)
+ texts = text_splitter.split_documents(texts)
+ else:
+ raise ValueError(
+ "Unsupported file type. Supported files are .txt and .pdf only."
+ )
+
+ embeddings = CohereEmbeddings(model=COHERE_MODEL_NAME)
+ return texts, embeddings
+
+
+# Database Initialization
+def initialize_database(texts, embeddings):
+ """
+ Initialize and populate a LanceDB database with documents and their embeddings.
+
+ Parameters:
+ - texts (list): A list of texts to be stored in the database.
+ - embeddings (CohereEmbeddings): An embeddings object used to generate vector embeddings for the texts.
+
+ Returns:
+ - LanceDB: An instance of LanceDB with the documents and their embeddings stored.
+ """
+ db = lancedb.connect(DB_PATH)
+ table = db.create_table(
+ "multiling-rag",
+ data=[
+ {
+ "vector": embeddings.embed_query("Hello World"),
+ "text": "Hello World",
+ "id": "1",
+ }
+ ],
+ mode="overwrite",
+ )
+ return LanceDB.from_documents(texts, embeddings, connection=table)
+
+
+# Translation Function
+def translate_text(text, from_code, to_code):
+ """
+ Translate a given text from one language to another.
+
+ Parameters:
+ - text (str): The text to translate.
+ - from_code (str): The ISO language code of the source language.
+ - to_code (str): The ISO language code of the target language.
+
+ Returns:
+ - str: The translated text.
+ """
+ try:
+ argostranslate.package.update_package_index()
+ available_packages = argostranslate.package.get_available_packages()
+ package_to_install = next(
+ filter(
+ lambda x: x.from_code == from_code and x.to_code == to_code,
+ available_packages,
+ )
+ )
+ argostranslate.package.install_from_path(package_to_install.download())
+ return argostranslate.translate.translate(text, from_code, to_code)
+ except Exception as e:
+ logger.error(f"Error in translate_text: {str(e)}")
+ return "Translation error"
+
+
+prompt_template = """Text: {context}
+
+Question: {question}
+
+Answer the question based on the text provided. If the text doesn't contain the answer, reply that the answer is not available."""
+PROMPT = PromptTemplate(
+ template=prompt_template, input_variables=["context", "question"]
+)
+
+
+# Question Answering Function
+def answer_question(question, input_language, output_language, db):
+ """
+ Answer a given question by retrieving relevant information from a database,
+ translating the question and answer if necessary.
+ Parameters:
+ - question (str): The question to answer.
+ - input_language (str): The language of the input question.
+ - output_language (str): The desired language of the answer.
+ - db (LanceDB): The LanceDB instance to use for information retrieval.
+
+ Returns:
+ - str: The answer to the question, in the desired output language
+ """
+ try:
+ input_lang_code = LANGUAGE_ISO_CODES[input_language]
+ output_lang_code = LANGUAGE_ISO_CODES[output_language]
+
+ question_in_english = (
+ translate_text(question, from_code=input_lang_code, to_code="en")
+ if input_language != "English"
+ else question
+ )
+ prompt = PromptTemplate(
+ template=prompt_template, input_variables=["context", "question"]
+ )
+ qa = RetrievalQA.from_chain_type(
+ llm=Cohere(model="command", temperature=0),
+ chain_type="stuff",
+ retriever=db.as_retriever(),
+ chain_type_kwargs={"prompt": prompt},
+ return_source_documents=True,
+ )
+
+ answer = qa({"query": question_in_english})
+ result_in_english = answer["result"].replace("\n", "").replace("Answer:", "")
+
+ return (
+ translate_text(result_in_english, from_code="en", to_code=output_lang_code)
+ if output_language != "English"
+ else result_in_english
+ )
+ except Exception as e:
+ logger.error(f"Error in answer_question: {str(e)}")
+ return "An error occurred while processing your question. Please try again."
+
+
+def setup_gradio_interface(db):
+ """
+ Setup a Gradio interface for interacting with the multilingual chatbot.
+
+ Parameters:
+ - db (LanceDB): The database instance to use for information retrieval.
+
+ Returns:
+ - gr.Interface: A Gradio interface object for the chatbot.
+ """
+
+ return gr.Interface(
+ fn=lambda question, input_language, output_language: answer_question(
+ question, input_language, output_language, db
+ ),
+ inputs=[
+ gr.Textbox(lines=2, placeholder="Type your question here..."),
+ gr.Dropdown(list(LANGUAGE_ISO_CODES.keys()), label="Input Language"),
+ gr.Dropdown(list(LANGUAGE_ISO_CODES.keys()), label="Output Language"),
+ ],
+ outputs="text",
+ title="Multilingual Chatbot",
+ description="Ask any question in your chosen language and get an answer in the language of your choice.",
+ )
+
+
+# Main Function
+def main():
+ INPUT_FILE_PATH = "healthy-diet-fact-sheet-394.pdf"
+ texts, embeddings = initialize_documents_and_embeddings(INPUT_FILE_PATH)
+ db = initialize_database(texts, embeddings)
+ iface = setup_gradio_interface(db)
+ iface.launch(share=True, debug=True)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/applications/Multilingual_RAG/requirements.txt b/applications/Multilingual_RAG/requirements.txt
new file mode 100644
index 0000000..f8e08ae
--- /dev/null
+++ b/applications/Multilingual_RAG/requirements.txt
@@ -0,0 +1,5 @@
+cohere
+langchain
+lancedb
+python-dotenv
+gradio
diff --git a/applications/chat_with_anywebsite/app.py b/applications/chat_with_anywebsite/app.py
index 8b8d076..7c5e746 100644
--- a/applications/chat_with_anywebsite/app.py
+++ b/applications/chat_with_anywebsite/app.py
@@ -13,7 +13,6 @@
class ChatbotHelper:
-
def __init__(self):
self.chatbot_instance = None
self.chat_history = []
@@ -135,7 +134,6 @@ def respond(self, message):
return bot_message
def run_interface(self):
-
iface = gr.Interface(
fn=self.respond,
title="Chatbot with URL or any website ",
diff --git a/applications/chat_with_anywebsite/main_app.ipynb b/applications/chat_with_anywebsite/main_app.ipynb
index 54226f9..699034a 100644
--- a/applications/chat_with_anywebsite/main_app.ipynb
+++ b/applications/chat_with_anywebsite/main_app.ipynb
@@ -79,7 +79,6 @@
"\n",
"\n",
"class ChatbotHelper:\n",
- "\n",
" def __init__(self):\n",
" self.chatbot_instance = None\n",
" self.chat_history = []\n",
@@ -207,7 +206,6 @@
" return bot_message\n",
"\n",
" def run_interface(self):\n",
- "\n",
" iface = gr.Interface(\n",
" fn=self.respond,\n",
" title=\"Chatbot with URL or any website \",\n",
diff --git a/applications/docchat-with-langroid/app.py b/applications/docchat-with-langroid/app.py
index d4f4a31..f92949d 100644
--- a/applications/docchat-with-langroid/app.py
+++ b/applications/docchat-with-langroid/app.py
@@ -11,7 +11,6 @@
uploadedFile = st.file_uploader("Choose a txt file")
if uploadedFile is not None:
-
with open(os.path.join("tempDir", uploadedFile.name), "wb") as f:
f.write(uploadedFile.getbuffer())
diff --git a/applications/docchat-with-langroid/utils.py b/applications/docchat-with-langroid/utils.py
index ea50cdc..3e7871c 100644
--- a/applications/docchat-with-langroid/utils.py
+++ b/applications/docchat-with-langroid/utils.py
@@ -41,7 +41,6 @@ def configure(filename):
def agent(cfg, prompt):
-
# Creating DocChatAgent
rag_agent = DocChatAgent(cfg)
diff --git a/applications/talk-with-github/.gitignore b/applications/talk-with-github/.gitignore
new file mode 100644
index 0000000..021c177
--- /dev/null
+++ b/applications/talk-with-github/.gitignore
@@ -0,0 +1,2 @@
+.lancedb
+example_data/
\ No newline at end of file
diff --git a/applications/talk-with-github/README.md b/applications/talk-with-github/README.md
new file mode 100644
index 0000000..4de5a32
--- /dev/null
+++ b/applications/talk-with-github/README.md
@@ -0,0 +1,41 @@
+# Talk to Github CodeSpaces using Qwen1.5
+
+Using this application, You can talk to Github Repositories. It will clone, embed all the Markdown, Python and Javascript files in the repository. This Application utilizes newly launched Qwen1.5 as a LLM.
+
+---
+**NOTE** | \n", - " | title | \n", - "summary | \n", - "authors | \n", - "url | \n", - "_distance | \n", - "
|---|---|---|---|---|---|
| 0 | \n", - "Unsupervised Learning via Meta-Learning | \n", - "A central goal of unsupervised learning is to acquire representations from\\nunlabeled data or experience that can be used for more effective learning of\\ndownstream tasks from modest amounts of labeled data. Many prior unsupervised\\nlearning works aim to do so by developing proxy objectives based on\\nreconstruction, disentanglement, prediction, and other metrics. Instead, we\\ndevelop an unsupervised meta-learning method that explicitly optimizes for the\\nability to learn a variety of tasks from small amounts of data. To do so, we\\nconstruct tasks from unlabeled data in an automatic way and run meta-learning\\nover the constructed tasks. Surprisingly, we find that, when integrated with\\nmeta-learning, relatively simple task construction mechanisms, such as\\nclustering embeddings, lead to good performance on a variety of downstream,\\nhuman-specified tasks. Our experiments across four image datasets indicate that\\nour unsupervised meta-learning approach acquires a learning algorithm without\\nany labeled data that is applicable to a wide range of downstream\\nclassification tasks, improving upon the embedding learned by four prior\\nunsupervised learning methods. | \n", - "[arxiv.Result.Author('Kyle Hsu'), arxiv.Result.Author('Sergey Levine'), arxiv.Result.Author('Chelsea Finn')] | \n", - "http://arxiv.org/abs/1810.02334v6 | \n", - "34.913574 | \n", - "
| 1 | \n", - "Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation | \n", - "Supervised deep learning-based methods yield accurate results for medical\\nimage segmentation. However, they require large labeled datasets for this, and\\nobtaining them is a laborious task that requires clinical expertise.\\nSemi/self-supervised learning-based approaches address this limitation by\\nexploiting unlabeled data along with limited annotated data. Recent\\nself-supervised learning methods use contrastive loss to learn good global\\nlevel representations from unlabeled images and achieve high performance in\\nclassification tasks on popular natural image datasets like ImageNet. In\\npixel-level prediction tasks such as segmentation, it is crucial to also learn\\ngood local level representations along with global representations to achieve\\nbetter accuracy. However, the impact of the existing local contrastive\\nloss-based methods remains limited for learning good local representations\\nbecause similar and dissimilar local regions are defined based on random\\naugmentations and spatial proximity; not based on the semantic label of local\\nregions due to lack of large-scale expert annotations in the\\nsemi/self-supervised setting. In this paper, we propose a local contrastive\\nloss to learn good pixel level features useful for segmentation by exploiting\\nsemantic label information obtained from pseudo-labels of unlabeled images\\nalongside limited annotated images. In particular, we define the proposed loss\\nto encourage similar representations for the pixels that have the same\\npseudo-label/ label while being dissimilar to the representation of pixels with\\ndifferent pseudo-label/label in the dataset. We perform pseudo-label based\\nself-training and train the network by jointly optimizing the proposed\\ncontrastive loss on both labeled and unlabeled sets and segmentation loss on\\nonly the limited labeled set. We evaluated on three public cardiac and prostate\\ndatasets, and obtain high segmentation performance. | \n", - "[arxiv.Result.Author('Krishna Chaitanya'), arxiv.Result.Author('Ertunc Erdil'), arxiv.Result.Author('Neerav Karani'), arxiv.Result.Author('Ender Konukoglu')] | \n", - "http://arxiv.org/abs/2112.09645v1 | \n", - "35.332321 | \n", - "
| 2 | \n", - "Universum GANs: Improving GANs through contradictions | \n", - "Limited availability of labeled-data makes any supervised learning problem\\nchallenging. Alternative learning settings like semi-supervised and universum\\nlearning alleviate the dependency on labeled data, but still require a large\\namount of unlabeled data, which may be unavailable or expensive to acquire.\\nGAN-based data generation methods have recently shown promise by generating\\nsynthetic samples to improve learning. However, most existing GAN based\\napproaches either provide poor discriminator performance under limited labeled\\ndata settings; or results in low quality generated data. In this paper, we\\npropose a Universum GAN game which provides improved discriminator accuracy\\nunder limited data settings, while generating high quality realistic data. We\\nfurther propose an evolving discriminator loss which improves its convergence\\nand generalization performance. We derive the theoretical guarantees and\\nprovide empirical results in support of our approach. | \n", - "[arxiv.Result.Author('Sauptik Dhar'), arxiv.Result.Author('Javad Heydari'), arxiv.Result.Author('Samarth Tripathi'), arxiv.Result.Author('Unmesh Kurup'), arxiv.Result.Author('Mohak Shah')] | \n", - "http://arxiv.org/abs/2106.09946v2 | \n", - "36.214127 | \n", - "
| \n", - " | title | \n", - "summary | \n", - "authors | \n", - "url | \n", - "score | \n", - "
|---|---|---|---|---|---|
| 0 | \n", - "Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation | \n", - "Supervised deep learning-based methods yield accurate results for medical\\nimage segmentation. However, they require large labeled datasets for this, and\\nobtaining them is a laborious task that requires clinical expertise.\\nSemi/self-supervised learning-based approaches address this limitation by\\nexploiting unlabeled data along with limited annotated data. Recent\\nself-supervised learning methods use contrastive loss to learn good global\\nlevel representations from unlabeled images and achieve high performance in\\nclassification tasks on popular natural image datasets like ImageNet. In\\npixel-level prediction tasks such as segmentation, it is crucial to also learn\\ngood local level representations along with global representations to achieve\\nbetter accuracy. However, the impact of the existing local contrastive\\nloss-based methods remains limited for learning good local representations\\nbecause similar and dissimilar local regions are defined based on random\\naugmentations and spatial proximity; not based on the semantic label of local\\nregions due to lack of large-scale expert annotations in the\\nsemi/self-supervised setting. In this paper, we propose a local contrastive\\nloss to learn good pixel level features useful for segmentation by exploiting\\nsemantic label information obtained from pseudo-labels of unlabeled images\\nalongside limited annotated images. In particular, we define the proposed loss\\nto encourage similar representations for the pixels that have the same\\npseudo-label/ label while being dissimilar to the representation of pixels with\\ndifferent pseudo-label/label in the dataset. We perform pseudo-label based\\nself-training and train the network by jointly optimizing the proposed\\ncontrastive loss on both labeled and unlabeled sets and segmentation loss on\\nonly the limited labeled set. We evaluated on three public cardiac and prostate\\ndatasets, and obtain high segmentation performance. | \n", - "[arxiv.Result.Author('Krishna Chaitanya'), arxiv.Result.Author('Ertunc Erdil'), arxiv.Result.Author('Neerav Karani'), arxiv.Result.Author('Ender Konukoglu')] | \n", - "http://arxiv.org/abs/2112.09645v1 | \n", - "7.476211 | \n", - "
| 1 | \n", - "Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells | \n", - "Unsupervised text encoding models have recently fueled substantial progress\\nin NLP. The key idea is to use neural networks to convert words in texts to\\nvector space representations based on word positions in a sentence and their\\ncontexts, which are suitable for end-to-end training of downstream tasks. We\\nsee a strikingly similar situation in spatial analysis, which focuses on\\nincorporating both absolute positions and spatial contexts of geographic\\nobjects such as POIs into models. A general-purpose representation model for\\nspace is valuable for a multitude of tasks. However, no such general model\\nexists to date beyond simply applying discretization or feed-forward nets to\\ncoordinates, and little effort has been put into jointly modeling distributions\\nwith vastly different characteristics, which commonly emerges from GIS data.\\nMeanwhile, Nobel Prize-winning Neuroscience research shows that grid cells in\\nmammals provide a multi-scale periodic representation that functions as a\\nmetric for location encoding and is critical for recognizing places and for\\npath-integration. Therefore, we propose a representation learning model called\\nSpace2Vec to encode the absolute positions and spatial relationships of places.\\nWe conduct experiments on two real-world geographic data for two different\\ntasks: 1) predicting types of POIs given their positions and context, 2) image\\nclassification leveraging their geo-locations. Results show that because of its\\nmulti-scale representations, Space2Vec outperforms well-established ML\\napproaches such as RBF kernels, multi-layer feed-forward nets, and tile\\nembedding approaches for location modeling and image classification tasks.\\nDetailed analysis shows that all baselines can at most well handle distribution\\nat one scale but show poor performances in other scales. In contrast,\\nSpace2Vec's multi-scale representation can handle distributions at different\\nscales. | \n", - "[arxiv.Result.Author('Gengchen Mai'), arxiv.Result.Author('Krzysztof Janowicz'), arxiv.Result.Author('Bo Yan'), arxiv.Result.Author('Rui Zhu'), arxiv.Result.Author('Ling Cai'), arxiv.Result.Author('Ni Lao')] | \n", - "http://arxiv.org/abs/2003.00824v1 | \n", - "5.525813 | \n", - "
| 2 | \n", - "Extending the WILDS Benchmark for Unsupervised Adaptation | \n", - "Machine learning systems deployed in the wild are often trained on a source\\ndistribution but deployed on a different target distribution. Unlabeled data\\ncan be a powerful point of leverage for mitigating these distribution shifts,\\nas it is frequently much more available than labeled data and can often be\\nobtained from distributions beyond the source distribution as well. However,\\nexisting distribution shift benchmarks with unlabeled data do not reflect the\\nbreadth of scenarios that arise in real-world applications. In this work, we\\npresent the WILDS 2.0 update, which extends 8 of the 10 datasets in the WILDS\\nbenchmark of distribution shifts to include curated unlabeled data that would\\nbe realistically obtainable in deployment. These datasets span a wide range of\\napplications (from histology to wildlife conservation), tasks (classification,\\nregression, and detection), and modalities (photos, satellite images,\\nmicroscope slides, text, molecular graphs). The update maintains consistency\\nwith the original WILDS benchmark by using identical labeled training,\\nvalidation, and test sets, as well as the evaluation metrics. On these\\ndatasets, we systematically benchmark state-of-the-art methods that leverage\\nunlabeled data, including domain-invariant, self-training, and self-supervised\\nmethods, and show that their success on WILDS is limited. To facilitate method\\ndevelopment and evaluation, we provide an open-source package that automates\\ndata loading and contains all of the model architectures and methods used in\\nthis paper. Code and leaderboards are available at https://wilds.stanford.edu. | \n", - "[arxiv.Result.Author('Shiori Sagawa'), arxiv.Result.Author('Pang Wei Koh'), arxiv.Result.Author('Tony Lee'), arxiv.Result.Author('Irena Gao'), arxiv.Result.Author('Sang Michael Xie'), arxiv.Result.Author('Kendrick Shen'), arxiv.Result.Author('Ananya Kumar'), arxiv.Result.Author('Weihua Hu'), arxiv.Result.Author('Michihiro Yasunaga'), arxiv.Result.Author('Henrik Marklund'), arxiv.Result.Author('Sara Beery'), arxiv.Result.Author('Etienne David'), arxiv.Result.Author('Ian Stavness'), arxiv.Result.Author('Wei Guo'), arxiv.Result.Author('Jure Leskovec'), arxiv.Result.Author('Kate Saenko'), arxiv.Result.Author('Tatsunori Hashimoto'), arxiv.Result.Author('Sergey Levine'), arxiv.Result.Author('Chelsea Finn'), arxiv.Result.Author('Percy Liang')] | \n", - "http://arxiv.org/abs/2112.05090v2 | \n", - "4.826763 | \n", - "
| 3 | \n", - "Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective | \n", - "We are used to the availability of big data generated in nearly all fields of\\nscience as a consequence of technological progress. However, the analysis of\\nsuch data possess vast challenges. One of these relates to the explainability\\nof artificial intelligence (AI) or machine learning methods. Currently, many of\\nsuch methods are non-transparent with respect to their working mechanism and\\nfor this reason are called black box models, most notably deep learning\\nmethods. However, it has been realized that this constitutes severe problems\\nfor a number of fields including the health sciences and criminal justice and\\narguments have been brought forward in favor of an explainable AI. In this\\npaper, we do not assume the usual perspective presenting explainable AI as it\\nshould be, but rather we provide a discussion what explainable AI can be. The\\ndifference is that we do not present wishful thinking but reality grounded\\nproperties in relation to a scientific theory beyond physics. | \n", - "[arxiv.Result.Author('Frank Emmert-Streib'), arxiv.Result.Author('Olli Yli-Harja'), arxiv.Result.Author('Matthias Dehmer')] | \n", - "http://arxiv.org/abs/2001.09464v1 | \n", - "4.770155 | \n", - "
| 4 | \n", - "Robustness of Generalized Learning Vector Quantization Models against Adversarial Attacks | \n", - "Adversarial attacks and the development of (deep) neural networks robust\\nagainst them are currently two widely researched topics. The robustness of\\nLearning Vector Quantization (LVQ) models against adversarial attacks has\\nhowever not yet been studied to the same extent. We therefore present an\\nextensive evaluation of three LVQ models: Generalized LVQ, Generalized Matrix\\nLVQ and Generalized Tangent LVQ. The evaluation suggests that both Generalized\\nLVQ and Generalized Tangent LVQ have a high base robustness, on par with the\\ncurrent state-of-the-art in robust neural network methods. In contrast to this,\\nGeneralized Matrix LVQ shows a high susceptibility to adversarial attacks,\\nscoring consistently behind all other models. Additionally, our numerical\\nevaluation indicates that increasing the number of prototypes per class\\nimproves the robustness of the models. | \n", - "[arxiv.Result.Author('Sascha Saralajew'), arxiv.Result.Author('Lars Holdijk'), arxiv.Result.Author('Maike Rees'), arxiv.Result.Author('Thomas Villmann')] | \n", - "http://arxiv.org/abs/1902.00577v2 | \n", - "4.715786 | \n", - "
| 5 | \n", - "Concept Whitening for Interpretable Image Recognition | \n", - "What does a neural network encode about a concept as we traverse through the\\nlayers? Interpretability in machine learning is undoubtedly important, but the\\ncalculations of neural networks are very challenging to understand. Attempts to\\nsee inside their hidden layers can either be misleading, unusable, or rely on\\nthe latent space to possess properties that it may not have. In this work,\\nrather than attempting to analyze a neural network posthoc, we introduce a\\nmechanism, called concept whitening (CW), to alter a given layer of the network\\nto allow us to better understand the computation leading up to that layer. When\\na concept whitening module is added to a CNN, the axes of the latent space are\\naligned with known concepts of interest. By experiment, we show that CW can\\nprovide us a much clearer understanding for how the network gradually learns\\nconcepts over layers. CW is an alternative to a batch normalization layer in\\nthat it normalizes, and also decorrelates (whitens) the latent space. CW can be\\nused in any layer of the network without hurting predictive performance. | \n", - "[arxiv.Result.Author('Zhi Chen'), arxiv.Result.Author('Yijie Bei'), arxiv.Result.Author('Cynthia Rudin')] | \n", - "http://arxiv.org/abs/2002.01650v5 | \n", - "4.590379 | \n", - "
| 6 | \n", - "General Cyclical Training of Neural Networks | \n", - "This paper describes the principle of \"General Cyclical Training\" in machine\\nlearning, where training starts and ends with \"easy training\" and the \"hard\\ntraining\" happens during the middle epochs. We propose several manifestations\\nfor training neural networks, including algorithmic examples (via\\nhyper-parameters and loss functions), data-based examples, and model-based\\nexamples. Specifically, we introduce several novel techniques: cyclical weight\\ndecay, cyclical batch size, cyclical focal loss, cyclical softmax temperature,\\ncyclical data augmentation, cyclical gradient clipping, and cyclical\\nsemi-supervised learning. In addition, we demonstrate that cyclical weight\\ndecay, cyclical softmax temperature, and cyclical gradient clipping (as three\\nexamples of this principle) are beneficial in the test accuracy performance of\\na trained model. Furthermore, we discuss model-based examples (such as\\npretraining and knowledge distillation) from the perspective of general\\ncyclical training and recommend some changes to the typical training\\nmethodology. In summary, this paper defines the general cyclical training\\nconcept and discusses several specific ways in which this concept can be\\napplied to training neural networks. In the spirit of reproducibility, the code\\nused in our experiments is available at \\url{https://github.com/lnsmith54/CFL}. | \n", - "[arxiv.Result.Author('Leslie N. Smith')] | \n", - "http://arxiv.org/abs/2202.08835v2 | \n", - "4.346595 | \n", - "
| 7 | \n", - "Explaining Aviation Safety Incidents Using Deep Temporal Multiple Instance Learning | \n", - "Although aviation accidents are rare, safety incidents occur more frequently\\nand require a careful analysis to detect and mitigate risks in a timely manner.\\nAnalyzing safety incidents using operational data and producing event-based\\nexplanations is invaluable to airline companies as well as to governing\\norganizations such as the Federal Aviation Administration (FAA) in the United\\nStates. However, this task is challenging because of the complexity involved in\\nmining multi-dimensional heterogeneous time series data, the lack of\\ntime-step-wise annotation of events in a flight, and the lack of scalable tools\\nto perform analysis over a large number of events. In this work, we propose a\\nprecursor mining algorithm that identifies events in the multidimensional time\\nseries that are correlated with the safety incident. Precursors are valuable to\\nsystems health and safety monitoring and in explaining and forecasting safety\\nincidents. Current methods suffer from poor scalability to high dimensional\\ntime series data and are inefficient in capturing temporal behavior. We propose\\nan approach by combining multiple-instance learning (MIL) and deep recurrent\\nneural networks (DRNN) to take advantage of MIL's ability to learn using weakly\\nsupervised data and DRNN's ability to model temporal behavior. We describe the\\nalgorithm, the data, the intuition behind taking a MIL approach, and a\\ncomparative analysis of the proposed algorithm with baseline models. We also\\ndiscuss the application to a real-world aviation safety problem using data from\\na commercial airline company and discuss the model's abilities and\\nshortcomings, with some final remarks about possible deployment directions. | \n", - "[arxiv.Result.Author('Vijay Manikandan Janakiraman')] | \n", - "http://arxiv.org/abs/1710.04749v2 | \n", - "3.825403 | \n", - "
| 8 | \n", - "Continual Unsupervised Representation Learning | \n", - "Continual learning aims to improve the ability of modern learning systems to\\ndeal with non-stationary distributions, typically by attempting to learn a\\nseries of tasks sequentially. Prior art in the field has largely considered\\nsupervised or reinforcement learning tasks, and often assumes full knowledge of\\ntask labels and boundaries. In this work, we propose an approach (CURL) to\\ntackle a more general problem that we will refer to as unsupervised continual\\nlearning. The focus is on learning representations without any knowledge about\\ntask identity, and we explore scenarios when there are abrupt changes between\\ntasks, smooth transitions from one task to another, or even when the data is\\nshuffled. The proposed approach performs task inference directly within the\\nmodel, is able to dynamically expand to capture new concepts over its lifetime,\\nand incorporates additional rehearsal-based techniques to deal with\\ncatastrophic forgetting. We demonstrate the efficacy of CURL in an unsupervised\\nlearning setting with MNIST and Omniglot, where the lack of labels ensures no\\ninformation is leaked about the task. Further, we demonstrate strong\\nperformance compared to prior art in an i.i.d setting, or when adapting the\\ntechnique to supervised tasks such as incremental class learning. | \n", - "[arxiv.Result.Author('Dushyant Rao'), arxiv.Result.Author('Francesco Visin'), arxiv.Result.Author('Andrei A. Rusu'), arxiv.Result.Author('Yee Whye Teh'), arxiv.Result.Author('Razvan Pascanu'), arxiv.Result.Author('Raia Hadsell')] | \n", - "http://arxiv.org/abs/1910.14481v1 | \n", - "3.443345 | \n", - "
| 9 | \n", - "Analysis of Generalizability of Deep Neural Networks Based on the Complexity of Decision Boundary | \n", - "For supervised learning models, the analysis of generalization ability\\n(generalizability) is vital because the generalizability expresses how well a\\nmodel will perform on unseen data. Traditional generalization methods, such as\\nthe VC dimension, do not apply to deep neural network (DNN) models. Thus, new\\ntheories to explain the generalizability of DNNs are required. In this study,\\nwe hypothesize that the DNN with a simpler decision boundary has better\\ngeneralizability by the law of parsimony (Occam's Razor). We create the\\ndecision boundary complexity (DBC) score to define and measure the complexity\\nof decision boundary of DNNs. The idea of the DBC score is to generate data\\npoints (called adversarial examples) on or near the decision boundary. Our new\\napproach then measures the complexity of the boundary using the entropy of\\neigenvalues of these data. The method works equally well for high-dimensional\\ndata. We use training data and the trained model to compute the DBC score. And,\\nthe ground truth for model's generalizability is its test accuracy. Experiments\\nbased on the DBC score have verified our hypothesis. The DBC is shown to\\nprovide an effective method to measure the complexity of a decision boundary\\nand gives a quantitative measure of the generalizability of DNNs. | \n", - "[arxiv.Result.Author('Shuyue Guan'), arxiv.Result.Author('Murray Loew')] | \n", - "http://arxiv.org/abs/2009.07974v1 | \n", - "3.419930 | \n", - "
| \n", + " | title | \n", + "summary | \n", + "authors | \n", + "url | \n", + "_distance | \n", + "
|---|---|---|---|---|---|
| 0 | \n", + "XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification | \n", + "In recent years, there have been numerous developments towards solving\\nmultimodal tasks, aiming to learn a stronger representation than through a\\nsingle modality. Certain aspects of the data can be particularly useful in this\\ncase - for example, correlations in the space or time domain across modalities\\n- but should be wisely exploited in order to benefit from their full predictive\\npotential. We propose two deep learning architectures with multimodal\\ncross-connections that allow for dataflow between several feature extractors\\n(XFlow). Our models derive more interpretable features and achieve better\\nperformances than models which do not exchange representations, usefully\\nexploiting correlations between audio and visual data, which have a different\\ndimensionality and are nontrivially exchangeable. Our work improves on existing\\nmultimodal deep learning algorithms in two essential ways: (1) it presents a\\nnovel method for performing cross-modality (before features are learned from\\nindividual modalities) and (2) extends the previously proposed\\ncross-connections which only transfer information between streams that process\\ncompatible data. Illustrating some of the representations learned by the\\nconnections, we analyse their contribution to the increase in discrimination\\nability and reveal their compatibility with a lip-reading network intermediate\\nrepresentation. We provide the research community with Digits, a new dataset\\nconsisting of three data types extracted from videos of people saying the\\ndigits 0-9. Results show that both cross-modal architectures outperform their\\nbaselines (by up to 11.5%) when evaluated on the AVletters, CUAVE and Digits\\ndatasets, achieving state-of-the-art results. | \n", + "[arxiv.Result.Author('Cătălina Cangea'), arxiv.Result.Author('Petar Veličković'), arxiv.Result.Author('Pietro Liò')] | \n", + "http://arxiv.org/abs/1709.00572v2 | \n", + "40.346901 | \n", + "
| 1 | \n", + "Dualing GANs | \n", + "Generative adversarial nets (GANs) are a promising technique for modeling a\\ndistribution from samples. It is however well known that GAN training suffers\\nfrom instability due to the nature of its maximin formulation. In this paper,\\nwe explore ways to tackle the instability problem by dualizing the\\ndiscriminator. We start from linear discriminators in which case conjugate\\nduality provides a mechanism to reformulate the saddle point objective into a\\nmaximization problem, such that both the generator and the discriminator of\\nthis 'dualing GAN' act in concert. We then demonstrate how to extend this\\nintuition to non-linear formulations. For GANs with linear discriminators our\\napproach is able to remove the instability in training, while for GANs with\\nnonlinear discriminators our approach provides an alternative to the commonly\\nused GAN training algorithm. | \n", + "[arxiv.Result.Author('Yujia Li'), arxiv.Result.Author('Alexander Schwing'), arxiv.Result.Author('Kuan-Chieh Wang'), arxiv.Result.Author('Richard Zemel')] | \n", + "http://arxiv.org/abs/1706.06216v1 | \n", + "40.449284 | \n", + "
| 2 | \n", + "Domain Generalization for Object Recognition with Multi-task Autoencoders | \n", + "The problem of domain generalization is to take knowledge acquired from a\\nnumber of related domains where training data is available, and to then\\nsuccessfully apply it to previously unseen domains. We propose a new feature\\nlearning algorithm, Multi-Task Autoencoder (MTAE), that provides good\\ngeneralization performance for cross-domain object recognition.\\n Our algorithm extends the standard denoising autoencoder framework by\\nsubstituting artificially induced corruption with naturally occurring\\ninter-domain variability in the appearance of objects. Instead of\\nreconstructing images from noisy versions, MTAE learns to transform the\\noriginal image into analogs in multiple related domains. It thereby learns\\nfeatures that are robust to variations across domains. The learnt features are\\nthen used as inputs to a classifier.\\n We evaluated the performance of the algorithm on benchmark image recognition\\ndatasets, where the task is to learn features from multiple datasets and to\\nthen predict the image label from unseen datasets. We found that (denoising)\\nMTAE outperforms alternative autoencoder-based models as well as the current\\nstate-of-the-art algorithms for domain generalization. | \n", + "[arxiv.Result.Author('Muhammad Ghifary'), arxiv.Result.Author('W. Bastiaan Kleijn'), arxiv.Result.Author('Mengjie Zhang'), arxiv.Result.Author('David Balduzzi')] | \n", + "http://arxiv.org/abs/1508.07680v1 | \n", + "41.127644 | \n", + "
| \n", + " | title | \n", + "summary | \n", + "authors | \n", + "url | \n", + "_distance | \n", + "
|---|---|---|---|---|---|
| 0 | \n", + "A General Theory for Training Learning Machine | \n", + "Though the deep learning is pushing the machine learning to a new stage,\\nbasic theories of machine learning are still limited. The principle of\\nlearning, the role of the a prior knowledge, the role of neuron bias, and the\\nbasis for choosing neural transfer function and cost function, etc., are still\\nfar from clear. In this paper, we present a general theoretical framework for\\nmachine learning. We classify the prior knowledge into common and\\nproblem-dependent parts, and consider that the aim of learning is to maximally\\nincorporate them. The principle we suggested for maximizing the former is the\\ndesign risk minimization principle, while the neural transfer function, the\\ncost function, as well as pretreatment of samples, are endowed with the role\\nfor maximizing the latter. The role of the neuron bias is explained from a\\ndifferent angle. We develop a Monte Carlo algorithm to establish the\\ninput-output responses, and we control the input-output sensitivity of a\\nlearning machine by controlling that of individual neurons. Applications of\\nfunction approaching and smoothing, pattern recognition and classification, are\\nprovided to illustrate how to train general learning machines based on our\\ntheory and algorithm. Our method may in addition induce new applications, such\\nas the transductive inference. | \n", + "[arxiv.Result.Author('Hong Zhao')] | \n", + "http://arxiv.org/abs/1704.06885v1 | \n", + "33.708359 | \n", + "
| 1 | \n", + "Learning Visual Reasoning Without Strong Priors | \n", + "Achieving artificial visual reasoning - the ability to answer image-related\\nquestions which require a multi-step, high-level process - is an important step\\ntowards artificial general intelligence. This multi-modal task requires\\nlearning a question-dependent, structured reasoning process over images from\\nlanguage. Standard deep learning approaches tend to exploit biases in the data\\nrather than learn this underlying structure, while leading methods learn to\\nvisually reason successfully but are hand-crafted for reasoning. We show that a\\ngeneral-purpose, Conditional Batch Normalization approach achieves\\nstate-of-the-art results on the CLEVR Visual Reasoning benchmark with a 2.4%\\nerror rate. We outperform the next best end-to-end method (4.5%) and even\\nmethods that use extra supervision (3.1%). We probe our model to shed light on\\nhow it reasons, showing it has learned a question-dependent, multi-step\\nprocess. Previous work has operated under the assumption that visual reasoning\\ncalls for a specialized architecture, but we show that a general architecture\\nwith proper conditioning can learn to visually reason effectively. | \n", + "[arxiv.Result.Author('Ethan Perez'), arxiv.Result.Author('Harm de Vries'), arxiv.Result.Author('Florian Strub'), arxiv.Result.Author('Vincent Dumoulin'), arxiv.Result.Author('Aaron Courville')] | \n", + "http://arxiv.org/abs/1707.03017v5 | \n", + "36.282284 | \n", + "
| 2 | \n", + "Encoder Based Lifelong Learning | \n", + "This paper introduces a new lifelong learning solution where a single model\\nis trained for a sequence of tasks. The main challenge that vision systems face\\nin this context is catastrophic forgetting: as they tend to adapt to the most\\nrecently seen task, they lose performance on the tasks that were learned\\npreviously. Our method aims at preserving the knowledge of the previous tasks\\nwhile learning a new one by using autoencoders. For each task, an\\nunder-complete autoencoder is learned, capturing the features that are crucial\\nfor its achievement. When a new task is presented to the system, we prevent the\\nreconstructions of the features with these autoencoders from changing, which\\nhas the effect of preserving the information on which the previous tasks are\\nmainly relying. At the same time, the features are given space to adjust to the\\nmost recent environment as only their projection into a low dimension\\nsubmanifold is controlled. The proposed system is evaluated on image\\nclassification tasks and shows a reduction of forgetting over the\\nstate-of-the-art | \n", + "[arxiv.Result.Author('Amal Rannen Triki'), arxiv.Result.Author('Rahaf Aljundi'), arxiv.Result.Author('Mathew B. Blaschko'), arxiv.Result.Author('Tinne Tuytelaars')] | \n", + "http://arxiv.org/abs/1704.01920v1 | \n", + "37.254250 | \n", + "
| \n", + " | title | \n", + "summary | \n", + "authors | \n", + "url | \n", + "score | \n", + "
|---|---|---|---|---|---|
| 0 | \n", + "Expert Gate: Lifelong Learning with a Network of Experts | \n", + "In this paper we introduce a model of lifelong learning, based on a Network\\nof Experts. New tasks / experts are learned and added to the model\\nsequentially, building on what was learned before. To ensure scalability of\\nthis process,data from previous tasks cannot be stored and hence is not\\navailable when learning a new task. A critical issue in such context, not\\naddressed in the literature so far, relates to the decision which expert to\\ndeploy at test time. We introduce a set of gating autoencoders that learn a\\nrepresentation for the task at hand, and, at test time, automatically forward\\nthe test sample to the relevant expert. This also brings memory efficiency as\\nonly one expert network has to be loaded into memory at any given time.\\nFurther, the autoencoders inherently capture the relatedness of one task to\\nanother, based on which the most relevant prior model to be used for training a\\nnew expert, with finetuning or learning without-forgetting, can be selected. We\\nevaluate our method on image classification and video prediction problems. | \n", + "[arxiv.Result.Author('Rahaf Aljundi'), arxiv.Result.Author('Punarjay Chakravarty'), arxiv.Result.Author('Tinne Tuytelaars')] | \n", + "http://arxiv.org/abs/1611.06194v2 | \n", + "4.703215 | \n", + "
| 1 | \n", + "Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs | \n", + "The idea of computer vision as the Bayesian inverse problem to computer\\ngraphics has a long history and an appealing elegance, but it has proved\\ndifficult to directly implement. Instead, most vision tasks are approached via\\ncomplex bottom-up processing pipelines. Here we show that it is possible to\\nwrite short, simple probabilistic graphics programs that define flexible\\ngenerative models and to automatically invert them to interpret real-world\\nimages. Generative probabilistic graphics programs consist of a stochastic\\nscene generator, a renderer based on graphics software, a stochastic likelihood\\nmodel linking the renderer's output and the data, and latent variables that\\nadjust the fidelity of the renderer and the tolerance of the likelihood model.\\nRepresentations and algorithms from computer graphics, originally designed to\\nproduce high-quality images, are instead used as the deterministic backbone for\\nhighly approximate and stochastic generative models. This formulation combines\\nprobabilistic programming, computer graphics, and approximate Bayesian\\ncomputation, and depends only on general-purpose, automatic inference\\ntechniques. We describe two applications: reading sequences of degraded and\\nadversarially obscured alphanumeric characters, and inferring 3D road models\\nfrom vehicle-mounted camera images. Each of the probabilistic graphics programs\\nwe present relies on under 20 lines of probabilistic code, and supports\\naccurate, approximately Bayesian inferences about ambiguous real-world images. | \n", + "[arxiv.Result.Author('Vikash K. Mansinghka'), arxiv.Result.Author('Tejas D. Kulkarni'), arxiv.Result.Author('Yura N. Perov'), arxiv.Result.Author('Joshua B. Tenenbaum')] | \n", + "http://arxiv.org/abs/1307.0060v1 | \n", + "4.515473 | \n", + "
| 2 | \n", + "Learning Visual Reasoning Without Strong Priors | \n", + "Achieving artificial visual reasoning - the ability to answer image-related\\nquestions which require a multi-step, high-level process - is an important step\\ntowards artificial general intelligence. This multi-modal task requires\\nlearning a question-dependent, structured reasoning process over images from\\nlanguage. Standard deep learning approaches tend to exploit biases in the data\\nrather than learn this underlying structure, while leading methods learn to\\nvisually reason successfully but are hand-crafted for reasoning. We show that a\\ngeneral-purpose, Conditional Batch Normalization approach achieves\\nstate-of-the-art results on the CLEVR Visual Reasoning benchmark with a 2.4%\\nerror rate. We outperform the next best end-to-end method (4.5%) and even\\nmethods that use extra supervision (3.1%). We probe our model to shed light on\\nhow it reasons, showing it has learned a question-dependent, multi-step\\nprocess. Previous work has operated under the assumption that visual reasoning\\ncalls for a specialized architecture, but we show that a general architecture\\nwith proper conditioning can learn to visually reason effectively. | \n", + "[arxiv.Result.Author('Ethan Perez'), arxiv.Result.Author('Harm de Vries'), arxiv.Result.Author('Florian Strub'), arxiv.Result.Author('Vincent Dumoulin'), arxiv.Result.Author('Aaron Courville')] | \n", + "http://arxiv.org/abs/1707.03017v5 | \n", + "4.332870 | \n", + "
| 3 | \n", + "Memory Aware Synapses: Learning what (not) to forget | \n", + "Humans can learn in a continuous manner. Old rarely utilized knowledge can be\\noverwritten by new incoming information while important, frequently used\\nknowledge is prevented from being erased. In artificial learning systems,\\nlifelong learning so far has focused mainly on accumulating knowledge over\\ntasks and overcoming catastrophic forgetting. In this paper, we argue that,\\ngiven the limited model capacity and the unlimited new information to be\\nlearned, knowledge has to be preserved or erased selectively. Inspired by\\nneuroplasticity, we propose a novel approach for lifelong learning, coined\\nMemory Aware Synapses (MAS). It computes the importance of the parameters of a\\nneural network in an unsupervised and online manner. Given a new sample which\\nis fed to the network, MAS accumulates an importance measure for each parameter\\nof the network, based on how sensitive the predicted output function is to a\\nchange in this parameter. When learning a new task, changes to important\\nparameters can then be penalized, effectively preventing important knowledge\\nrelated to previous tasks from being overwritten. Further, we show an\\ninteresting connection between a local version of our method and Hebb's\\nrule,which is a model for the learning process in the brain. We test our method\\non a sequence of object recognition tasks and on the challenging problem of\\nlearning an embedding for predicting $<$subject, predicate, object$>$ triplets.\\nWe show state-of-the-art performance and, for the first time, the ability to\\nadapt the importance of the parameters based on unlabeled data towards what the\\nnetwork needs (not) to forget, which may vary depending on test conditions. | \n", + "[arxiv.Result.Author('Rahaf Aljundi'), arxiv.Result.Author('Francesca Babiloni'), arxiv.Result.Author('Mohamed Elhoseiny'), arxiv.Result.Author('Marcus Rohrbach'), arxiv.Result.Author('Tinne Tuytelaars')] | \n", + "http://arxiv.org/abs/1711.09601v4 | \n", + "4.307245 | \n", + "
| 4 | \n", + "Explaining Aviation Safety Incidents Using Deep Temporal Multiple Instance Learning | \n", + "Although aviation accidents are rare, safety incidents occur more frequently\\nand require a careful analysis to detect and mitigate risks in a timely manner.\\nAnalyzing safety incidents using operational data and producing event-based\\nexplanations is invaluable to airline companies as well as to governing\\norganizations such as the Federal Aviation Administration (FAA) in the United\\nStates. However, this task is challenging because of the complexity involved in\\nmining multi-dimensional heterogeneous time series data, the lack of\\ntime-step-wise annotation of events in a flight, and the lack of scalable tools\\nto perform analysis over a large number of events. In this work, we propose a\\nprecursor mining algorithm that identifies events in the multidimensional time\\nseries that are correlated with the safety incident. Precursors are valuable to\\nsystems health and safety monitoring and in explaining and forecasting safety\\nincidents. Current methods suffer from poor scalability to high dimensional\\ntime series data and are inefficient in capturing temporal behavior. We propose\\nan approach by combining multiple-instance learning (MIL) and deep recurrent\\nneural networks (DRNN) to take advantage of MIL's ability to learn using weakly\\nsupervised data and DRNN's ability to model temporal behavior. We describe the\\nalgorithm, the data, the intuition behind taking a MIL approach, and a\\ncomparative analysis of the proposed algorithm with baseline models. We also\\ndiscuss the application to a real-world aviation safety problem using data from\\na commercial airline company and discuss the model's abilities and\\nshortcomings, with some final remarks about possible deployment directions. | \n", + "[arxiv.Result.Author('Vijay Manikandan Janakiraman')] | \n", + "http://arxiv.org/abs/1710.04749v2 | \n", + "4.206257 | \n", + "
| 5 | \n", + "A General Theory for Training Learning Machine | \n", + "Though the deep learning is pushing the machine learning to a new stage,\\nbasic theories of machine learning are still limited. The principle of\\nlearning, the role of the a prior knowledge, the role of neuron bias, and the\\nbasis for choosing neural transfer function and cost function, etc., are still\\nfar from clear. In this paper, we present a general theoretical framework for\\nmachine learning. We classify the prior knowledge into common and\\nproblem-dependent parts, and consider that the aim of learning is to maximally\\nincorporate them. The principle we suggested for maximizing the former is the\\ndesign risk minimization principle, while the neural transfer function, the\\ncost function, as well as pretreatment of samples, are endowed with the role\\nfor maximizing the latter. The role of the neuron bias is explained from a\\ndifferent angle. We develop a Monte Carlo algorithm to establish the\\ninput-output responses, and we control the input-output sensitivity of a\\nlearning machine by controlling that of individual neurons. Applications of\\nfunction approaching and smoothing, pattern recognition and classification, are\\nprovided to illustrate how to train general learning machines based on our\\ntheory and algorithm. Our method may in addition induce new applications, such\\nas the transductive inference. | \n", + "[arxiv.Result.Author('Hong Zhao')] | \n", + "http://arxiv.org/abs/1704.06885v1 | \n", + "4.150894 | \n", + "
| 6 | \n", + "A Brief Survey of Deep Reinforcement Learning | \n", + "Deep reinforcement learning is poised to revolutionise the field of AI and\\nrepresents a step towards building autonomous systems with a higher level\\nunderstanding of the visual world. Currently, deep learning is enabling\\nreinforcement learning to scale to problems that were previously intractable,\\nsuch as learning to play video games directly from pixels. Deep reinforcement\\nlearning algorithms are also applied to robotics, allowing control policies for\\nrobots to be learned directly from camera inputs in the real world. In this\\nsurvey, we begin with an introduction to the general field of reinforcement\\nlearning, then progress to the main streams of value-based and policy-based\\nmethods. Our survey will cover central algorithms in deep reinforcement\\nlearning, including the deep $Q$-network, trust region policy optimisation, and\\nasynchronous advantage actor-critic. In parallel, we highlight the unique\\nadvantages of deep neural networks, focusing on visual understanding via\\nreinforcement learning. To conclude, we describe several current areas of\\nresearch within the field. | \n", + "[arxiv.Result.Author('Kai Arulkumaran'), arxiv.Result.Author('Marc Peter Deisenroth'), arxiv.Result.Author('Miles Brundage'), arxiv.Result.Author('Anil Anthony Bharath')] | \n", + "http://arxiv.org/abs/1708.05866v2 | \n", + "3.549962 | \n", + "
| 7 | \n", + "Interpretable Explanations of Black Boxes by Meaningful Perturbation | \n", + "As machine learning algorithms are increasingly applied to high impact yet\\nhigh risk tasks, such as medical diagnosis or autonomous driving, it is\\ncritical that researchers can explain how such algorithms arrived at their\\npredictions. In recent years, a number of image saliency methods have been\\ndeveloped to summarize where highly complex neural networks \"look\" in an image\\nfor evidence for their predictions. However, these techniques are limited by\\ntheir heuristic nature and architectural constraints. In this paper, we make\\ntwo main contributions: First, we propose a general framework for learning\\ndifferent kinds of explanations for any black box algorithm. Second, we\\nspecialise the framework to find the part of an image most responsible for a\\nclassifier decision. Unlike previous works, our method is model-agnostic and\\ntestable because it is grounded in explicit and interpretable image\\nperturbations. | \n", + "[arxiv.Result.Author('Ruth Fong'), arxiv.Result.Author('Andrea Vedaldi')] | \n", + "http://arxiv.org/abs/1704.03296v4 | \n", + "3.451381 | \n", + "
| 8 | \n", + "Self corrective Perturbations for Semantic Segmentation and Classification | \n", + "Convolutional Neural Networks have been a subject of great importance over\\nthe past decade and great strides have been made in their utility for producing\\nstate of the art performance in many computer vision problems. However, the\\nbehavior of deep networks is yet to be fully understood and is still an active\\narea of research. In this work, we present an intriguing behavior: pre-trained\\nCNNs can be made to improve their predictions by structurally perturbing the\\ninput. We observe that these perturbations - referred as Guided Perturbations -\\nenable a trained network to improve its prediction performance without any\\nlearning or change in network weights. We perform various ablative experiments\\nto understand how these perturbations affect the local context and feature\\nrepresentations. Furthermore, we demonstrate that this idea can improve\\nperformance of several existing approaches on semantic segmentation and scene\\nlabeling tasks on the PASCAL VOC dataset and supervised classification tasks on\\nMNIST and CIFAR10 datasets. | \n", + "[arxiv.Result.Author('Swami Sankaranarayanan'), arxiv.Result.Author('Arpit Jain'), arxiv.Result.Author('Ser Nam Lim')] | \n", + "http://arxiv.org/abs/1703.07928v2 | \n", + "3.417501 | \n", + "
| 9 | \n", + "Graph Approximation and Clustering on a Budget | \n", + "We consider the problem of learning from a similarity matrix (such as\\nspectral clustering and lowd imensional embedding), when computing pairwise\\nsimilarities are costly, and only a limited number of entries can be observed.\\nWe provide a theoretical analysis using standard notions of graph\\napproximation, significantly generalizing previous results (which focused on\\nspectral clustering with two clusters). We also propose a new algorithmic\\napproach based on adaptive sampling, which experimentally matches or improves\\non previous methods, while being considerably more general and computationally\\ncheaper. | \n", + "[arxiv.Result.Author('Ethan Fetaya'), arxiv.Result.Author('Ohad Shamir'), arxiv.Result.Author('Shimon Ullman')] | \n", + "http://arxiv.org/abs/1406.2602v1 | \n", + "3.358721 | \n", + "
+
+## Using Python file
+### Install Requirements
```bash
-pip install git+https://github.com/lancedb/lancedb.git@main#subdirectory=python
+pip install -r requirements.txt
```
-Install other dependecies from requirements.txt file of this folder.
+### Run
+```
+python3 main.py
+```
diff --git a/examples/instruct-multitask/main.ipynb b/examples/instruct-multitask/main.ipynb
index fa05037..2771386 100644
--- a/examples/instruct-multitask/main.ipynb
+++ b/examples/instruct-multitask/main.ipynb
@@ -1,8147 +1,8042 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "58b30d1f-28e7-4e27-b256-f7c7bbcfbf4b",
- "metadata": {
- "id": "58b30d1f-28e7-4e27-b256-f7c7bbcfbf4b"
- },
- "source": [
- "## InstructOR - A multitask custom embedding model for task based applications, made easier with LanceDB\n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "id": "2506f81b-cf52-41f2-9f0d-6b20836b15b6",
- "metadata": {
- "id": "2506f81b-cf52-41f2-9f0d-6b20836b15b6"
- },
- "source": [
- "### Installing all dependencies"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "id": "d8c13226-8a68-485a-86ba-6243eb1ed26e",
- "metadata": {
- "id": "d8c13226-8a68-485a-86ba-6243eb1ed26e",
- "outputId": "3b783914-5300-4586-cb66-73a2a931ec51",
- "colab": {
- "base_uri": "https://localhost:8080/"
- }
- },
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "Collecting lancedb\n",
- " Downloading lancedb-0.5.0-py3-none-any.whl (87 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.4/87.4 kB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hCollecting deprecation (from lancedb)\n",
- " Downloading deprecation-2.1.0-py2.py3-none-any.whl (11 kB)\n",
- "Collecting pylance==0.9.6 (from lancedb)\n",
- " Downloading pylance-0.9.6-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.6 MB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m18.6/18.6 MB\u001b[0m \u001b[31m44.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hCollecting ratelimiter~=1.0 (from lancedb)\n",
- " Downloading ratelimiter-1.2.0.post0-py3-none-any.whl (6.6 kB)\n",
- "Collecting retry>=0.9.2 (from lancedb)\n",
- " Downloading retry-0.9.2-py2.py3-none-any.whl (8.0 kB)\n",
- "Requirement already satisfied: tqdm>=4.27.0 in /usr/local/lib/python3.10/dist-packages (from lancedb) (4.66.1)\n",
- "Requirement already satisfied: pydantic>=1.10 in /usr/local/lib/python3.10/dist-packages (from lancedb) (1.10.13)\n",
- "Requirement already satisfied: attrs>=21.3.0 in /usr/local/lib/python3.10/dist-packages (from lancedb) (23.2.0)\n",
- "Collecting semver>=3.0 (from lancedb)\n",
- " Downloading semver-3.0.2-py3-none-any.whl (17 kB)\n",
- "Requirement already satisfied: cachetools in /usr/local/lib/python3.10/dist-packages (from lancedb) (5.3.2)\n",
- "Requirement already satisfied: pyyaml>=6.0 in /usr/local/lib/python3.10/dist-packages (from lancedb) (6.0.1)\n",
- "Requirement already satisfied: click>=8.1.7 in /usr/local/lib/python3.10/dist-packages (from lancedb) (8.1.7)\n",
- "Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from lancedb) (2.31.0)\n",
- "Collecting overrides>=0.7 (from lancedb)\n",
- " Downloading overrides-7.6.0-py3-none-any.whl (17 kB)\n",
- "Collecting pyarrow>=12 (from pylance==0.9.6->lancedb)\n",
- " Downloading pyarrow-15.0.0-cp310-cp310-manylinux_2_28_x86_64.whl (38.3 MB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m38.3/38.3 MB\u001b[0m \u001b[31m12.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hRequirement already satisfied: numpy>=1.22 in /usr/local/lib/python3.10/dist-packages (from pylance==0.9.6->lancedb) (1.23.5)\n",
- "Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10->lancedb) (4.5.0)\n",
- "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->lancedb) (3.3.2)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->lancedb) (3.6)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->lancedb) (2.0.7)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->lancedb) (2023.11.17)\n",
- "Requirement already satisfied: decorator>=3.4.2 in /usr/local/lib/python3.10/dist-packages (from retry>=0.9.2->lancedb) (4.4.2)\n",
- "Collecting py<2.0.0,>=1.4.26 (from retry>=0.9.2->lancedb)\n",
- " Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m98.7/98.7 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from deprecation->lancedb) (23.2)\n",
- "Installing collected packages: ratelimiter, semver, pyarrow, py, overrides, deprecation, retry, pylance, lancedb\n",
- " Attempting uninstall: pyarrow\n",
- " Found existing installation: pyarrow 10.0.1\n",
- " Uninstalling pyarrow-10.0.1:\n",
- " Successfully uninstalled pyarrow-10.0.1\n",
- "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
- "ibis-framework 7.1.0 requires pyarrow<15,>=2, but you have pyarrow 15.0.0 which is incompatible.\u001b[0m\u001b[31m\n",
- "\u001b[0mSuccessfully installed deprecation-2.1.0 lancedb-0.5.0 overrides-7.6.0 py-1.11.0 pyarrow-15.0.0 pylance-0.9.6 ratelimiter-1.2.0.post0 retry-0.9.2 semver-3.0.2\n"
- ]
- }
- ],
- "source": [
- "!pip install lancedb"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "id": "9bb0ec2f-9953-427b-b358-c9af1a78a61e",
- "metadata": {
- "id": "9bb0ec2f-9953-427b-b358-c9af1a78a61e",
- "outputId": "7253901e-611a-4a65-9ccb-177ebc353799",
- "colab": {
- "base_uri": "https://localhost:8080/"
- }
- },
- "outputs": [
+ "cells": [
{
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "Collecting InstructorEmbedding\n",
- " Downloading InstructorEmbedding-1.0.1-py2.py3-none-any.whl (19 kB)\n",
- "Collecting sentence-transformers\n",
- " Downloading sentence-transformers-2.2.2.tar.gz (85 kB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.0/86.0 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
- "Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.1.0+cu121)\n",
- "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (1.5.3)\n",
- "Requirement already satisfied: transformers<5.0.0,>=4.6.0 in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (4.35.2)\n",
- "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (4.66.1)\n",
- "Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (0.16.0+cu121)\n",
- "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (1.23.5)\n",
- "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (1.2.2)\n",
- "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (1.11.4)\n",
- "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (3.8.1)\n",
- "Collecting sentencepiece (from sentence-transformers)\n",
- " Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
- "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m9.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
- "\u001b[?25hRequirement already satisfied: huggingface-hub>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from sentence-transformers) (0.20.2)\n",
- "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch) (3.13.1)\n",
- "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch) (4.5.0)\n",
- "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch) (1.12)\n",
- "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.2.1)\n",
- "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.3)\n",
- "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch) (2023.6.0)\n",
- "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch) (2.1.0)\n",
- "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas) (2.8.2)\n",
- "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas) (2023.3.post1)\n",
- "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (2.31.0)\n",
- "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (6.0.1)\n",
- "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers) (23.2)\n",
- "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas) (1.16.0)\n",
- "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (2023.6.3)\n",
- "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (0.15.0)\n",
- "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (0.4.1)\n",
- "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (2.1.3)\n",
- "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk->sentence-transformers) (8.1.7)\n",
- "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->sentence-transformers) (1.3.2)\n",
- "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence-transformers) (3.2.0)\n",
- "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch) (1.3.0)\n",
- "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/dist-packages (from torchvision->sentence-transformers) (9.4.0)\n",
- "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (3.3.2)\n",
- "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (3.6)\n",
- "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (2.0.7)\n",
- "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (2023.11.17)\n",
- "Building wheels for collected packages: sentence-transformers\n",
- " Building wheel for sentence-transformers (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
- " Created wheel for sentence-transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125923 sha256=9d7322894ecb5581cb484d80da823d44ed5d996c5f936d11179e7f83ce2b0128\n",
- " Stored in directory: /root/.cache/pip/wheels/62/f2/10/1e606fd5f02395388f74e7462910fe851042f97238cbbd902f\n",
- "Successfully built sentence-transformers\n",
- "Installing collected packages: sentencepiece, InstructorEmbedding, sentence-transformers\n",
- "Successfully installed InstructorEmbedding-1.0.1 sentence-transformers-2.2.2 sentencepiece-0.1.99\n"
- ]
- }
- ],
- "source": [
- "!pip install InstructorEmbedding sentence-transformers torch pandas"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "b4ce6fbd-75fb-4663-b68f-716295ae3720",
- "metadata": {
- "id": "b4ce6fbd-75fb-4663-b68f-716295ae3720"
- },
- "source": [
- "If you want to calculate customized embeddings for specific sentences, you may follow the unified template to write instructions:\n",
- "\n",
- "\"Represent the [**domain**] [**text_type**] for [**task_objective**]:\"\n",
- "\n",
- "Here are some examples:\n",
- "\n",
- "- \"Represent the **Science** **sentence**:\"\n",
- "- \"Represent the **Financial** **statement**:\"\n",
- "- \"Represent the **Wikipedia** **document** for **retrieval**:\"\n",
- "- \"Represent the **Wikipedia** **question** for **retrieving supporting documents**:\""
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7be2cd8c-710f-4eca-b93e-2b5f5876198a",
- "metadata": {
- "id": "7be2cd8c-710f-4eca-b93e-2b5f5876198a"
- },
- "source": [
- "### Importing neccessary libraries"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "id": "63f36f93-cf85-4548-a66c-5e4c8b8f9401",
- "metadata": {
- "id": "63f36f93-cf85-4548-a66c-5e4c8b8f9401"
- },
- "outputs": [],
- "source": [
- "import lancedb\n",
- "from lancedb.pydantic import LanceModel, Vector\n",
- "from lancedb.embeddings import get_registry\n",
- "from lancedb.embeddings import InstructorEmbeddingFunction"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "c3258997-580c-4231-b578-65db806425ef",
- "metadata": {
- "id": "c3258997-580c-4231-b578-65db806425ef"
- },
- "source": [
- "### Calling the embedding model from LanceDB embedding's API"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "id": "9aaa6835-fa7c-4197-b86b-e25efd7a7eb3",
- "metadata": {
- "id": "9aaa6835-fa7c-4197-b86b-e25efd7a7eb3",
- "outputId": "c3e92570-a581-476b-8af2-3eda7a42d963",
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 604,
- "referenced_widgets": [
- "cc21d68060fd452ea3f6d88809fc8d83",
- "91662b908f594e8098f2a771253ca54d",
- "4952d1b0838243b8892413a428bf31e8",
- "0c43343050ae434da5283d8873cefb59",
- "6ddd08007bc542339f4f007db9295a80",
- "045dd5aad3ef4f3086b339c87ae1d4e0",
- "6b5b533336fe4be597ee9fada67b9361",
- "b7e407edc0924ac08bb63c5457577650",
- "93af466ed85a471989c8d4ca7548689e",
- "1050f2ffb343420dab3c337bba19b67a",
- "be157869076542ceaadba08de34b92b4",
- "dc9b5a5b6eff4d089a3e6617a058d12d",
- "1475ef22bc9e4a75a5df037be93aee3a",
- "dc765b75716640f4afcedda301134d4e",
- "134519853c8747ea8712e57ed3ae3314",
- "cef497b8e277432b9eb67866092ee3ab",
- "c78f12921d474cc08192202d21d7a393",
- "2e77d0ee827547578e5cf668074b3461",
- "bedf89a7e9eb4ded96ffe36cc9efb62a",
- "2a2f5bce9d8645ee87dcd78759f02e86",
- "09b49fdbbf464423942ceaae81c73a2a",
- "8fbda417e40d497b8cd5b34dbbebf056",
- "8f0be5f370e848e48c8af0d16732fa8d",
- "21b7724f3a334091b420a76fb259673f",
- "1ba5a64ac3ad4d0daec31b5fae2558d9",
- "31c935361aa64872b250436b11e22e25",
- "11f4093cd59141daac246bd4502b8386",
- "2e8e3c2378d74134ab5f52458898625d",
- "a159aedd48734230adb551cc566e2bb6",
- "6d71fa6f3254458e8043ccaef184259f",
- "685818435e4e4854bc4db30901646dbd",
- "ac6a16abd9344b5a81bbbd23d937451a",
- "6e6b5e9f990149fd9777feeed6a50b9e",
- "7df2cf7718cc4b4e9474a6899d7aebed",
- "4d2f7690daf74d979dfba56070cff25c",
- "2320d116079f40949b96838dea810256",
- "110fc0e3a2b34ad59b3e3f1079db5634",
- "1da66756abac408aad0ba9aa00d06edb",
- "35e7cf4e37144e7ca30e987d8640fbd8",
- "c88d76f1f0744662adef3c984a02bd51",
- "f969049f905d4506b075a1f458879aed",
- "4dafb3bba3244d3198d4874c652ab392",
- "41f9f81181e94b738aa8519e06e0eb98",
- "58d5ae3db4744114a46c10875e20fc57",
- "0c84050a77784e1c957abe54e2e7136d",
- "2513d673fedf4e2a99a7cd1eae60e3e6",
- "1b24ab2ea6a240189cfefbb366a148bf",
- "fdc51d49926d4480b0b3a3bf172d10bd",
- "bbf7d922302d4fd0a310288ddd625b37",
- "91a80cc8e3aa4e839cc15df54eb9e3a2",
- "97c494623fb7445c95b0344c7b6e8a0d",
- "5ee21a42325f4cf38ac3dc10f8f0d071",
- "c7d5c7f1f8e245a991eab17c2b348d4b",
- "8b832c750e4543a29add338babf1dee6",
- "08b8d4d9e6af447a984ab0a5a060e9b2",
- "e2f29a34f1cf4aaa93d6b295ab24651b",
- "ac0983ca73104accbbbdf839c201fd9b",
- "e71b7ed56c14485581903c43ff24814a",
- "56e4cc997f7c40d095d889332dcf05e9",
- "af4da2b121004f56a65e80d0db8cd752",
- "cbff90e2a4d4402d8cc41e03f91600d8",
- "c62f97274d1b44d28a1b6a3023d7d0fc",
- "3c2a936b1e8a4376a244f129d51bb585",
- "0838a8867f2547f0868cbe7e8bfa8972",
- "6cfae5931e474c58ba4b886b2a3ff91f",
- "c05fba47c731437c9f6a3c6a33cf5368",
- "c9b8f2fb8d0e4721af184d8ffa0d9727",
- "2fb0c6408328462b820e83ae302a6a12",
- "f6be3d7393f04189bde73a1a162cd8e9",
- "0eebe0a3d0254f159b3cd71c82712adc",
- "67a25967a35a43b7ad4d76388e87346e",
- "f4856859239246478abc6b78d76faaa0",
- "adeb1a8b67fd40fdab22d34dab683584",
- "04b1bb0130584ecda9fb500f6fb29cbb",
- "a67bc84da01943288a3df82a24e10009",
- "66c47f80c4aa4ec49bb50056e889a058",
- "088eecbfd94740ea911dd7783807357f",
- "a34a0e508a844b23ab2ece4117d259cc",
- "1a866a412c4e47849ff9d20740b18bb1",
- "352cbe6e176c4c6ba6ba2bc0b73ee582",
- "c91fe51c38cc4bc099dfb4ce3a228a53",
- "5a517aa90bc041db8dbc0b5b9da8983d",
- "e9e91349ce454aad9f1bf64d7fc99a94",
- "7e69c9d2678e412ebbe98aeb1c8860ab",
- "b0d21e795a4541fba84e83fd1bd98bfc",
- "ed2c5b9a57574f748f7f19d4e72c308a",
- "4dd72427b97f432285f9a61966fbf449",
- "f30d5649bc1640b890e134845b0cf467",
- "cbb81eda043440fca55a8e5029600bef",
- "1f1a346862b4472c9016bdd5a67831ce",
- "05b3e2e86c2c45efa037aa1af4a94c0e",
- "8133bad5ad044004af46da228f1df18f",
- "5cf6ca7d1d4143a09fa8143f9a92c20e",
- "24e6e0ffcb554afea1e0ecfa1697f7ae",
- "21627b3436b5478e9719bca8aa362e8c",
- "de24aee2524e4f7dbf19527cb9a3a5ab",
- "bba0c3e06cba43fda7ec81043f314afa",
- "4de01e80d28b416b9b376c71bb10a764",
- "a417b5f89618400785940f156111241b",
- "923b9816a28845cb8a7efb4c9e045794",
- "517ed74de73945caab4fef79a7099248",
- "36bab1ad881f4e3c99ca9c87a0ffce4e",
- "a170d20815a64a8092f699df75096a60",
- "5a5976bf89ee46829f23b966a4d01310",
- "2f6b9ef37d2541e8a50fa44a2ef679c0",
- "f2d8ac28550e459099d3fc35a5e0c44f",
- "6e71210bd41a47cfb74bca5e038a4c27",
- "a45514c69ade4ed18d76ff0a17161332",
- "db1fccc71f8e486ea736ab7af814b80f",
- "f52c0c7be2e4495d969633f01a7448b7",
- "4f8ba6dc8853443cad2b17b60a2ebb3e",
- "b7ba5729b4184c05b59fc71d3da2b8cf",
- "1059a5f96c9a4e86a4cd3de2ae637f11",
- "d824c380c1f14ad7bcba1f9f96482804",
- "70328803a86c4f45bb1460e6b85af148",
- "b500c2c72b604c018732314aa89c3ed7",
- "215e0e9134ec4507bc6b520edc4b12e2",
- "d7ada020d7844a4ab097b247e1fd2eca",
- "4f93f304c2f34c8da2f1ac3d63f42792",
- "499151538bfa4e2b89cb63975915f232",
- "9f2549e6b3f744e595d394af3ff5d9f6",
- "8c4358e53b4b4bf8989df31a89f5b98c",
- "6b31457fb7d1476083341a8b4ca1bcc1",
- "835fcac3d7264fc4bf1bdb852b753319",
- "f158aa268b9f43edbefa421b42cbd811",
- "f298ba6549e946f5b6bbcd834a2bb62c",
- "3f1e1cc51c0e44acacf5439ef7318c9e",
- "b7b7cb30233242caa48b502bc5e9aa50",
- "ef43dd6867dc44068bbf430c538a570a",
- "7d42ba8e8eb645dd8ee3be06fa521519",
- "5710b5dea81e4b689e7b578688a6d422",
- "1cbd2169fcc8416fbb695abe65c5077c",
- "da58ba8b291b471186b7bdcfe1a1b569",
- "3084c899dce244beb40b560562becb6e",
- "f4146a247e9f47ccbcb3acfd0716cb3d",
- "b8eaa9617108450288bb426837a83bf7",
- "2631837c1d1842979459aea74dc0ea28",
- "a98cb6bb5b704137af9b4306bdbcbd19",
- "15a313a94ef44c85b7e0426de64644d2",
- "f322e3638d37411f8167f4105d56c2ee",
- "1650fe6d44534dc2a11e6484fef96e83",
- "c72c1aa55dba44ca9907284d43520b29",
- "f604384a01044fe3a004faa7745b54bf",
- "9c4138054bf14563bb673a06d7a42079",
- "4ea2abf8251a4c7f8eda895f537f21a2",
- "74a74a4d8fd540568cb0d39fef64c344",
- "d801dfa29781422ca1725b91f28788c5",
- "8255d2debd754fcda5cf5564931b319a",
- "69acb9a8574e4bdd923781e1957cb473",
- "05e201a25c124c228fae91b177635c61",
- "9d47ad7be4e14591b9a48c1d4cd69f37",
- "8a0ee973221b44d3b3cbd9b06525b891",
- "61392336c50844a4930d36f103545d2c",
- "11ebd5986953482cb0785e0a9c069b1c"
- ]
- }
- },
- "outputs": [
- {
- "output_type": "stream",
- "name": "stderr",
- "text": [
- "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
- "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
- "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
- "You will be able to reuse this secret in all of your notebooks.\n",
- "Please note that authentication is recommended but still optional to access public models or datasets.\n",
- " warnings.warn(\n"
- ]
+ "cell_type": "markdown",
+ "id": "58b30d1f-28e7-4e27-b256-f7c7bbcfbf4b",
+ "metadata": {
+ "id": "58b30d1f-28e7-4e27-b256-f7c7bbcfbf4b"
+ },
+ "source": [
+ "## InstructOR - A multitask custom embedding model for task based applications, made easier with LanceDB\n",
+ ""
+ ]
},
{
- "output_type": "display_data",
- "data": {
- "text/plain": [
- ".gitattributes: 0%| | 0.00/1.48k [00:00] 5.70G 62.1MB/s in 2m 5s \n",
- "\n",
- "2024-01-23 17:08:37 (46.7 MB/s) - ‘diffusiondb_lance.tar.gz’ saved [6121107645/6121107645]\n",
- "\n",
- "diffusiondb_test/\n",
- "diffusiondb_test/_versions/\n",
- "diffusiondb_test/_latest.manifest\n",
- "diffusiondb_test/data/\n",
- "diffusiondb_test/data/138fc0d8-a806-4b10-84f8-00dc381afdad.lance\n",
- "diffusiondb_test/_versions/1.manifest\n"
- ]
- }
- ],
- "source": [
- "!wget https://eto-public.s3.us-west-2.amazonaws.com/datasets/diffusiondb_lance.tar.gz\n",
- "!tar -xvf diffusiondb_lance.tar.gz\n",
- "!mv diffusiondb_test rawdata.lance"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e2fcbf61",
- "metadata": {
- "id": "e2fcbf61"
- },
- "source": [
- "## Create / Open LanceDB Table"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "id": "b3317a3c",
- "metadata": {
- "id": "b3317a3c"
- },
- "outputs": [],
- "source": [
- "import pyarrow.compute as pc\n",
- "import lance\n",
- "\n",
- "db = lancedb.connect(\"~/datasets/demo\")\n",
- "if \"diffusiondb\" in db.table_names():\n",
- " tbl = db.open_table(\"diffusiondb\")\n",
- "else:\n",
- " # First data processing and full-text-search index\n",
- " data = lance.dataset(\"rawdata.lance\").to_table()\n",
- " # remove null prompts\n",
- " # tbl = db.create_table(\"diffusiondb\", data.filter(~pc.field(\"prompt\").is_null()), mode=\"overwrite\") # OOM\n",
- " tbl = db.create_table(\"diffusiondb\", data, mode=\"overwrite\")\n",
- " tbl.create_fts_index([\"prompt\"])"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d7e4fc03",
- "metadata": {
- "id": "d7e4fc03"
- },
- "source": [
- "## Create CLIP embedding function for the text"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "id": "f8331d87",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "f8331d87",
- "outputId": "54d77a42-340d-42f4-e890-38512af7525f"
- },
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
- "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
- "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
- "You will be able to reuse this secret in all of your notebooks.\n",
- "Please note that authentication is recommended but still optional to access public models or datasets.\n",
- " warnings.warn(\n"
- ]
- }
- ],
- "source": [
- "from transformers import CLIPModel, CLIPProcessor, CLIPTokenizerFast\n",
- "\n",
- "MODEL_ID = \"openai/clip-vit-base-patch32\"\n",
- "\n",
- "tokenizer = CLIPTokenizerFast.from_pretrained(MODEL_ID)\n",
- "model = CLIPModel.from_pretrained(MODEL_ID)\n",
- "processor = CLIPProcessor.from_pretrained(MODEL_ID)\n",
- "\n",
- "\n",
- "def embed_func(query):\n",
- " inputs = tokenizer([query], padding=True, return_tensors=\"pt\")\n",
- " text_features = model.get_text_features(**inputs)\n",
- " return text_features.detach().numpy()[0]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "id": "82c50eaf",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "82c50eaf",
- "outputId": "bbae933a-9652-48ec-f7f9-c479f7a24118"
- },
- "outputs": [
- {
- "data": {
- "text/plain": [
- "prompt: string\n",
- "seed: uint32\n",
- "step: uint16\n",
- "cfg: float\n",
- "sampler: string\n",
- "width: uint16\n",
- "height: uint16\n",
- "timestamp: timestamp[s]\n",
- "image_nsfw: float\n",
- "prompt_nsfw: float\n",
- "vector: fixed_size_list
+Colab walkthrough -
### Get dataset
```bash
diff --git a/examples/multimodal_clip_diffusiondb/main.ipynb b/examples/multimodal_clip_diffusiondb/main.ipynb
new file mode 100644
index 0000000..a208f70
--- /dev/null
+++ b/examples/multimodal_clip_diffusiondb/main.ipynb
@@ -0,0 +1,3394 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "c11fde21",
+ "metadata": {
+ "id": "c11fde21"
+ },
+ "source": [
+ "# Multimodal search using CLIP\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "06c53ccb-5654-4150-93a2-cdf9ff4d8d26",
+ "metadata": {
+ "id": "06c53ccb-5654-4150-93a2-cdf9ff4d8d26"
+ },
+ "source": [
+ "### Installing all dependencies"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "69fb1627",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "69fb1627",
+ "outputId": "7516efba-c93a-4201-ad7a-0d3c4d82d0c1"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Requirement already satisfied: pip in /usr/local/lib/python3.10/dist-packages (23.1.2)\n",
+ "Requirement already satisfied: install in /usr/local/lib/python3.10/dist-packages (1.3.5)\n",
+ "Requirement already satisfied: tantivy==0.20.1 in /usr/local/lib/python3.10/dist-packages (0.20.1)\n"
+ ]
+ }
+ ],
+ "source": [
+ "!pip install --quiet -U lancedb\n",
+ "!pip install --quiet gradio==3.41.2 transformers torch torchvision duckdb\n",
+ "!pip install pip install pip install tantivy==0.20.1"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2d53ade3",
+ "metadata": {
+ "id": "2d53ade3"
+ },
+ "source": [
+ "## First run setup: Download data and pre-process\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "9b7e97f9",
+ "metadata": {
+ "id": "9b7e97f9"
+ },
+ "outputs": [],
+ "source": [
+ "import io\n",
+ "import PIL\n",
+ "import duckdb\n",
+ "import lancedb"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "5ba75742",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "5ba75742",
+ "outputId": "c97f0590-3839-4f67-cb5f-60295142f99b"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "--2024-02-25 04:51:11-- https://eto-public.s3.us-west-2.amazonaws.com/datasets/diffusiondb_lance.tar.gz\n",
+ "Resolving eto-public.s3.us-west-2.amazonaws.com (eto-public.s3.us-west-2.amazonaws.com)... 3.5.79.102, 3.5.82.217, 3.5.77.120, ...\n",
+ "Connecting to eto-public.s3.us-west-2.amazonaws.com (eto-public.s3.us-west-2.amazonaws.com)|3.5.79.102|:443... connected.\n",
+ "HTTP request sent, awaiting response... 200 OK\n",
+ "Length: 6121107645 (5.7G) [application/x-gzip]\n",
+ "Saving to: ‘diffusiondb_lance.tar.gz’\n",
+ "\n",
+ "diffusiondb_lance.t 100%[===================>] 5.70G 69.8MB/s in 88s \n",
+ "\n",
+ "2024-02-25 04:52:39 (66.6 MB/s) - ‘diffusiondb_lance.tar.gz’ saved [6121107645/6121107645]\n",
+ "\n",
+ "diffusiondb_test/\n",
+ "diffusiondb_test/_versions/\n",
+ "diffusiondb_test/_latest.manifest\n",
+ "diffusiondb_test/data/\n",
+ "diffusiondb_test/data/138fc0d8-a806-4b10-84f8-00dc381afdad.lance\n",
+ "diffusiondb_test/_versions/1.manifest\n"
+ ]
+ }
+ ],
+ "source": [
+ "!wget https://eto-public.s3.us-west-2.amazonaws.com/datasets/diffusiondb_lance.tar.gz\n",
+ "!tar -xvf diffusiondb_lance.tar.gz\n",
+ "!mv diffusiondb_test rawdata.lance"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2fcbf61",
+ "metadata": {
+ "id": "e2fcbf61"
+ },
+ "source": [
+ "## Create / Open LanceDB Table"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "b3317a3c",
+ "metadata": {
+ "id": "b3317a3c"
+ },
+ "outputs": [],
+ "source": [
+ "import pyarrow.compute as pc\n",
+ "import lance\n",
+ "\n",
+ "db = lancedb.connect(\"~/datasets/demo\")\n",
+ "if \"diffusiondb\" in db.table_names():\n",
+ " tbl = db.open_table(\"diffusiondb\")\n",
+ "else:\n",
+ " # First data processing and full-text-search index\n",
+ " data = lance.dataset(\"rawdata.lance\").to_table()\n",
+ " # remove null prompts\n",
+ " # tbl = db.create_table(\"diffusiondb\", data.filter(~pc.field(\"prompt\").is_null()), mode=\"overwrite\") # OOM\n",
+ " tbl = db.create_table(\"diffusiondb\", data, mode=\"overwrite\")\n",
+ " tbl.create_fts_index([\"prompt\"])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d7e4fc03",
+ "metadata": {
+ "id": "d7e4fc03"
+ },
+ "source": [
+ "## Create CLIP embedding function for the text"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "f8331d87",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 439,
+ "referenced_widgets": [
+ "ce95ce4a0e5346debb0a5f9202214e51",
+ "029f3597c525493893877e3ed4105951",
+ "31fc28c33c2a43d6a9f3edfd687a06aa",
+ "1952bb6499d247018b3bb8987f4c09bd",
+ "fb28818f86144e4b96bffa9f78f22d0e",
+ "9cebd2e362d342a8bce9b28f46bd35e2",
+ "0e317fbb6dd8470992e912e419a28185",
+ "a913f319f6e246039b11500884328720",
+ "fdfc01b44f0841dabd221d68752d3d72",
+ "ab8d874c0a184ca2ba81114e3eaf7409",
+ "d8b428b1b0464413b4fe5a9d4f111c57",
+ "daacb1e1a2814655a7d09f65ff8452aa",
+ "cf61f81503ae4dc08fa09394f3f5cccd",
+ "ebeadf1ec0c1418499ddbde06f4996f7",
+ "d005ee7956a44436957aa90c3300424c",
+ "f95919ae1e9c49bf86b4a5af88e0c2da",
+ "46ad1889dcac48cdbebad4f05ef09e2b",
+ "cd263d8c5fd84a35a91d874699f5d51a",
+ "304da0a1d41b40ac972c8c290c643870",
+ "40cac08994364eb09a9cc2f9f0bde825",
+ "aa75ab10ee4f4f85bc83221d7cfd36b6",
+ "ac69897159f84312b11add482fcf6d07",
+ "b3ea9b5595e1427a802318a98b514203",
+ "a8e504801bf54ffb87d250e9321d6aa2",
+ "4cb8ea17edd645e1aa275602a4abf8e7",
+ "9d0cad1fdd564482bf011f7fef778d4b",
+ "25b23989c05e483bbe56cb6605785eb1",
+ "5eea8e0e481d4f34ae1366bd38ee4434",
+ "6ad1a3c9c73341c5a2390ca9dd62668d",
+ "1d189babe20742409805cb9fa555beae",
+ "76e8ff4bd1ac452ba8251dc1f3228dfd",
+ "0832ef0b231c4a078603ce3f43781d5f",
+ "dc9a73097e614bc18aa09ed968050a62",
+ "772c4c47a58543a8a1a178f40597920f",
+ "455e6832a03a4c1aa5aab48a9109d91b",
+ "0bc11bbd6038444cbef4721bfebbd531",
+ "6d853fdc57cd4f35b4d0b6385177e8b7",
+ "3c2a47c25bd142adaa5fde9e90d66151",
+ "a8b9d35c4e424ba9b8a0c6518ad6a1b4",
+ "7a92918e31854bf4bb6a3e595b7b1a07",
+ "f8e42c0f4f8c4a6d910387c58676f660",
+ "78138fe07e094f54a8f372ac3e7dd96f",
+ "f1d094606e374cf1a86c8855c421ce01",
+ "6f3b8a6c0f9a4f33ae3b14d34fc2d4ac",
+ "d9e17ca730e34e44b78bd6a3a5307c7a",
+ "73e2ca308ae441e68c632dae241da9b0",
+ "124a9c8793f340fab919e90bcdcfd208",
+ "776cf021de614b668bdecf6c2f22bc07",
+ "5cabe7eba19f4fe5a6ebb0a8e019130f",
+ "320d8f855cdf4223a7efa9c11a9c7d4b",
+ "98a3539fa99c44f09444a05f65a88a33",
+ "08784f5ae8d0488980e188dceaf85172",
+ "e08c37c9b5cf4431b8ac3275f3884ddc",
+ "428c0c75d4d94234bd6eb0515e602dba",
+ "5ed4f7ff855440b28d8c0cb443260022",
+ "8fe30c2025df48bd825ea00bbf623326",
+ "6f37e35f95a14b5ea1046e672c491c52",
+ "95923a6c40944ac1b8cf917d8572478a",
+ "10107106ec1e4fd4b91736b7a9c985c5",
+ "20d453f0fd2c44bd81a1883bc7584074",
+ "085bcef04fa94ecb8fcf6df74602c8ca",
+ "ef32f6da77214a1caa24962a37e706ed",
+ "3f4e3aa5b1f34b518711148c3e1b1ec7",
+ "67fdff76b75c4703a483e8cbe0d5ea55",
+ "1846b4ef6dd94ac29d2ddf90e5a1ce6d",
+ "29ae5071deec41859a57e5cf6cb37ad5",
+ "eb38b4cae0e44591acfcedf07cc134cb",
+ "55d0ca31f24b48ef911ae02fb089c1ad",
+ "caa0ac3e4c664879a0cc0f6bd2663fa7",
+ "7e08e9d81ad24353886a5f26ac4d1f1c",
+ "f8250d657f5240efab2ba1725826ee72",
+ "5ed78c44b67f435e90b9abca36b86cb8",
+ "6b6b143e4e5045d6a5710886b3876b1c",
+ "ce033a7365e640c58f7835007655460b",
+ "1a4efbd30ac34718b8977845a66369f4",
+ "87445801dca54689ae4dbca714b4849e",
+ "19878e3fe3e3479e8bfbdd3f52a12d33",
+ "26904ea599a140f0b7ab4e74b2d1f608",
+ "65c145203efd4d7c9a3c1b8eb3a6d306",
+ "cd67afa173284b84a3d617b07d95f9d0",
+ "df2f32a0509b41f99e7f0c0fcb08ff35",
+ "a32279b40489404b842122ad4168a1a7",
+ "20943327f8ad4e818c22543ecc45a482",
+ "b6538a0a3b1043adb5cb5c7f07a57938",
+ "7cac0fb25dba43c38d609b7fe47bb5da",
+ "3bda882470684f8ea32ac008d86a7c58",
+ "1338c410755b41688a101be17dcb7a84",
+ "59d983edf3a641b1b442ed17128ca243"
+ ]
+ },
+ "id": "f8331d87",
+ "outputId": "8622151e-5547-4eec-df17-baf7c1d98c03"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
+ "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
+ "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
+ "You will be able to reuse this secret in all of your notebooks.\n",
+ "Please note that authentication is recommended but still optional to access public models or datasets.\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "display_data",
+ "data": {
+ "text/plain": [
+ "tokenizer_config.json: 0%| | 0.00/568 [00:00[512]\n",
+ " child 0, item: float\n",
+ "image: binary"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 7
+ }
+ ],
+ "source": [
+ "tbl.schema\n",
+ "# tbl.to_pandas().head() # OOM"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5e4d7a54",
+ "metadata": {
+ "id": "5e4d7a54"
+ },
+ "source": [
+ "\n",
+ "## Search functions for Gradio"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "id": "10b8de6d",
+ "metadata": {
+ "id": "10b8de6d"
+ },
+ "outputs": [],
+ "source": [
+ "def find_image_vectors(query):\n",
+ " emb = embed_func(query)\n",
+ " code = (\n",
+ " \"import lancedb\\n\"\n",
+ " \"db = lancedb.connect('~/datasets/demo')\\n\"\n",
+ " \"tbl = db.open_table('diffusiondb')\\n\\n\"\n",
+ " f\"embedding = embed_func('{query}')\\n\"\n",
+ " \"tbl.search(embedding).limit(9).to_df()\"\n",
+ " )\n",
+ " return (_extract(tbl.search(emb).limit(9).to_pandas()), code)\n",
+ "\n",
+ "\n",
+ "def find_image_keywords(query):\n",
+ " code = (\n",
+ " \"import lancedb\\n\"\n",
+ " \"db = lancedb.connect('~/datasets/demo')\\n\"\n",
+ " \"tbl = db.open_table('diffusiondb')\\n\\n\"\n",
+ " f\"tbl.search('{query}').limit(9).to_df()\"\n",
+ " )\n",
+ " return (_extract(tbl.search(query).limit(9).to_pandas()), code)\n",
+ "\n",
+ "\n",
+ "def find_image_sql(query):\n",
+ " code = (\n",
+ " \"import lancedb\\n\"\n",
+ " \"import duckdb\\n\"\n",
+ " \"db = lancedb.connect('~/datasets/demo')\\n\"\n",
+ " \"tbl = db.open_table('diffusiondb')\\n\\n\"\n",
+ " \"diffusiondb = tbl.to_lance()\\n\"\n",
+ " f\"duckdb.sql('{query}').to_df()\"\n",
+ " )\n",
+ " diffusiondb = tbl.to_lance()\n",
+ " return (_extract(duckdb.sql(query).to_df()), code)\n",
+ "\n",
+ "\n",
+ "def _extract(df):\n",
+ " image_col = \"image\"\n",
+ " return [\n",
+ " (PIL.Image.open(io.BytesIO(row[image_col])), row[\"prompt\"])\n",
+ " for _, row in df.iterrows()\n",
+ " ]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "61aaf19b",
+ "metadata": {
+ "id": "61aaf19b"
+ },
+ "source": [
+ "## Setup Gradio interface"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "id": "b6f40300",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 630
+ },
+ "id": "b6f40300",
+ "outputId": "1c9a7f57-a32d-42a7-a61f-b2fc1008c3b3"
+ },
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "
### Get dataset
diff --git a/examples/multimodal_video_search/README.md b/examples/multimodal_video_search/README.md
index 3923f84..d4389ba 100644
--- a/examples/multimodal_video_search/README.md
+++ b/examples/multimodal_video_search/README.md
@@ -2,7 +2,7 @@
We used LanceDB to store frames every thirty seconds and the title of 13000+ videos, 5 random from each top category from the Youtube 8M dataset.
Then, we used the CLIP model to embed frames and titles together. With LanceDB, we can perform embedding, keyword, and SQL search on these videos.
-
+
Colab walkthrough -
diff --git a/examples/product-recommender/lancedb_cloud/main.ipynb b/examples/product-recommender/lancedb_cloud/main.ipynb
index caa9149..f3c6f44 100644
--- a/examples/product-recommender/lancedb_cloud/main.ipynb
+++ b/examples/product-recommender/lancedb_cloud/main.ipynb
@@ -1491,7 +1491,6 @@
"outputs": [],
"source": [
"def products_bought_by_user_in_the_past(user_id: int, top: int = 10):\n",
- "\n",
" selected = data[data.user_id == user_id].sort_values(\n",
" by=[\"total_orders\"], ascending=False\n",
" )\n",
diff --git a/examples/product-recommender/main.ipynb b/examples/product-recommender/main.ipynb
index 2a802a7..66c5c68 100644
--- a/examples/product-recommender/main.ipynb
+++ b/examples/product-recommender/main.ipynb
@@ -1107,7 +1107,6 @@
"outputs": [],
"source": [
"def products_bought_by_user_in_the_past(user_id: int, top: int = 10):\n",
- "\n",
" selected = data[data.user_id == user_id].sort_values(\n",
" by=[\"total_orders\"], ascending=False\n",
" )\n",
diff --git a/examples/product-recommender/main.py b/examples/product-recommender/main.py
index ffcb218..776c04f 100644
--- a/examples/product-recommender/main.py
+++ b/examples/product-recommender/main.py
@@ -12,7 +12,6 @@
def products_bought_by_user_in_the_past(user_id: int, top: int = 10):
-
selected = data[data.user_id == user_id].sort_values(
by=["total_orders"], ascending=False
)
diff --git a/examples/search-within-images-with-sam-and-clip/main.ipynb b/examples/search-within-images-with-sam-and-clip/main.ipynb
index 5e2dfe8..f45ef59 100644
--- a/examples/search-within-images-with-sam-and-clip/main.ipynb
+++ b/examples/search-within-images-with-sam-and-clip/main.ipynb
@@ -533,7 +533,6 @@
"source": [
"# find the image using natural language query\n",
"def search_image_with_user_query(vector_table, img_id, user_query):\n",
- "\n",
" text = tokenizer(user_query)\n",
" k_embedding = model.encode_text(text).tolist() # Use tolist() instead of to_list()\n",
" # Flatten k_embedding to a List[float]\n",
diff --git a/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/README.md b/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/README.md
index e7ee9c1..a60f368 100644
--- a/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/README.md
+++ b/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/README.md
@@ -1,5 +1,7 @@
[](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/clip_text_image_search.ipynb)
+## Accelerate vector search Applications using OpenVINO
+
**CLIP from OpenAI for Text-to-Image and Image-to-Image searching** and we’ll also do a comparative analysis of the Pytorch model, FP16 OpenVINO format, and INT8 OpenVINO format in terms of speedup.
Here are a few Key points converted in this article.
@@ -12,4 +14,4 @@ Text-to-Image and Image-to-Image Search using CLIP
**These Results are on 13th Gen Intel(R) Core(TM) i5–13420H using OpenVINO=2023.2 and NNCF=2.7.0 version.**
-[Read More](https://blog.lancedb.com/accelerate-vector-search-applications-using-openvino-51366eabf866)
\ No newline at end of file
+[Read More in Blog](https://blog.lancedb.com/accelerate-vector-search-applications-using-openvino-51366eabf866)
\ No newline at end of file
diff --git a/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/clip_text_image_search.ipynb b/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/clip_text_image_search.ipynb
index 82ec6b1..87f62b2 100644
--- a/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/clip_text_image_search.ipynb
+++ b/tutorials/Accelerate-Vector-Search-Applications-Using-OpenVINO/clip_text_image_search.ipynb
@@ -537,7 +537,6 @@
"\n",
"\n",
"def get_image(image_URL):\n",
- "\n",
" response = requests.get(image_URL)\n",
" image = Image.open(BytesIO(response.content)).convert(\"RGB\")\n",
"\n",
@@ -545,7 +544,6 @@
"\n",
"\n",
"def get_image_caption(image_ID):\n",
- "\n",
" return image_data[image_ID][\"caption\"]"
]
},
@@ -1119,14 +1117,12 @@
"\n",
"\n",
"def plot_images(images):\n",
- "\n",
" for image in images:\n",
" plt.imshow(image)\n",
" plt.show()\n",
"\n",
"\n",
"def plot_images_by_side(top_images):\n",
- "\n",
" index_values = list(top_images.index.values)\n",
" list_images = [top_images.iloc[idx].image for idx in index_values]\n",
" list_captions = [top_images.iloc[idx].caption for idx in index_values]\n",
diff --git a/tutorials/Corrective-RAG-with_Langgraph/CRAG_with_Langgraph.ipynb b/tutorials/Corrective-RAG-with_Langgraph/CRAG_with_Langgraph.ipynb
new file mode 100644
index 0000000..a5ea7cc
--- /dev/null
+++ b/tutorials/Corrective-RAG-with_Langgraph/CRAG_with_Langgraph.ipynb
@@ -0,0 +1,805 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "d8Di5OGwUzRu"
+ },
+ "source": [
+ "## Corrective RAG(CRAG)\n",
+ "\n",
+ "Self-reflection can enhance RAG, enabling correction of poor quality retrieval or generations.\n",
+ "\n",
+ "**Corrective Retrieval-Augmented Generation (CRAG) is a method that works like a built-in fact-checker.**\n",
+ "\n",
+ "It adds both creativity and accuracy by creating text and then checking for any mistakes or made-up information. This helps make sure the final result is reliable and matches real-world facts. It's like a safety feature for AI writers, making their work more trustworthy and lowering the chances of spreading false information."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "bf_lVgcXfMM-"
+ },
+ "source": [
+ "Here we'll see how to implement ideas from the Corrective RAG (CRAG) [Paper](https://arxiv.org/pdf/2401.15884.pdf) using LangGraph."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "gUlaOeBxpIxD"
+ },
+ "outputs": [],
+ "source": [
+ "# installing dependencies (in quiet mode)\n",
+ "!pip install langchain_community tiktoken langchain-openai lancedb langchain langchainhub langgraph tavily-python sentence-transformers -q"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "4cOO-I77fjfx"
+ },
+ "source": [
+ "#### Set Tavily Search API Key for web search and OpenAI key\n",
+ "\n",
+ "Tavily is a search engine specifically designed for AI agents and large language models (LLMs).\n",
+ "\n",
+ "Get free credits [link](https://app.tavily.com/sign-in)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "d37-6ceEpVr8"
+ },
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "\n",
+ "os.environ[\"TAVILY_API_KEY\"] = \"tvly-...\"\n",
+ "os.environ[\"OPENAI_API_KEY\"] = \"sk-...\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "DVqIVS5BhZg-"
+ },
+ "source": [
+ "Corrective-RAG (CRAG) is a recent paper that talks about a cool way to make a self-reflective RAG.\n",
+ "\n",
+ "The method givesating/scores retrieved documents based on how well they answer a question:\n",
+ "\n",
+ "For Correct documents -\n",
+ "\n",
+ "1. If at least one document is really relevant, it moves on to creating text\n",
+ "2. Before creating text, it cleans up the knowledge\n",
+ "3. This breaks down the document into \"knowledge strips\"\n",
+ "4. It rates each strip and gets rid of ones that don't matter\n",
+ "\n",
+ "For Ambiguous or Incorrect documents -\n",
+ "1. If all documents are not relevant enough or if it's not sure, the method looks for more information\n",
+ "2. It uses a web search to add more details to what it found\n",
+ "3. The diagram in the paper also show that they might change the question to get better results.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "z3W48wXGmnwT"
+ },
+ "source": [
+ "### Building Retriever"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "f74S0SFHp_1s"
+ },
+ "outputs": [],
+ "source": [
+ "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
+ "from langchain_community.document_loaders import WebBaseLoader\n",
+ "from langchain_community.vectorstores import LanceDB\n",
+ "from langchain_openai import ChatOpenAI, OpenAIEmbeddings\n",
+ "\n",
+ "# Using Jay Alammer's articles on Transformers, Bert and using transformers for retrival\n",
+ "urls = [\n",
+ " \"https://jalammar.github.io/illustrated-transformer/\",\n",
+ " \"https://jalammar.github.io/illustrated-bert/\",\n",
+ " \"https://jalammar.github.io/illustrated-retrieval-transformer/\",\n",
+ "]\n",
+ "\n",
+ "docs = [WebBaseLoader(url).load() for url in urls]\n",
+ "docs_list = [item for sublist in docs for item in sublist]\n",
+ "\n",
+ "# document chunking\n",
+ "text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(\n",
+ " chunk_size=250, chunk_overlap=0\n",
+ ")\n",
+ "doc_splits = text_splitter.split_documents(docs_list)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "3z6nWU2ruYmD"
+ },
+ "outputs": [],
+ "source": [
+ "import lancedb\n",
+ "\n",
+ "\n",
+ "def lanceDBConnection(embed):\n",
+ " db = lancedb.connect(\"/tmp/lancedb\")\n",
+ " table = db.create_table(\n",
+ " \"crag_demo\",\n",
+ " data=[{\"vector\": embed.embed_query(\"Hello World\"), \"text\": \"Hello World\"}],\n",
+ " mode=\"overwrite\",\n",
+ " )\n",
+ "\n",
+ " return table"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "o63KVQsIuYAT"
+ },
+ "outputs": [],
+ "source": [
+ "# Huggingface embeddings\n",
+ "embedder = OpenAIEmbeddings()\n",
+ "# LanceDB as vector store\n",
+ "table = lanceDBConnection(embedder)\n",
+ "vectorstore = LanceDB.from_documents(\n",
+ " documents=doc_splits,\n",
+ " embedding=embedder,\n",
+ " connection=table,\n",
+ ")\n",
+ "\n",
+ "# ready with our retriever\n",
+ "retriever = vectorstore.as_retriever()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "CazkArNn1rTK"
+ },
+ "source": [
+ "### Defining Langgraph\n",
+ "We will define a graph for building Langgraph.\n",
+ "\n",
+ "We can access any Graph node as `state['keys']`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "F09RgJ1tqLNg"
+ },
+ "outputs": [],
+ "source": [
+ "from typing import Dict, TypedDict\n",
+ "\n",
+ "from langchain_core.messages import BaseMessage\n",
+ "\n",
+ "\n",
+ "class GraphState(TypedDict):\n",
+ " \"\"\"\n",
+ " Represents the state of our graph.\n",
+ "\n",
+ " Attributes:\n",
+ " keys: A dictionary where each key is a string.\n",
+ " \"\"\"\n",
+ "\n",
+ " keys: Dict[str, any]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "qSpy-ZriDDXf"
+ },
+ "source": [
+ "### Defining Nodes for Langgraph"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "JEdzCdNPUN-I"
+ },
+ "source": [
+ "### Nodes and Edges\n",
+ "\n",
+ "Each Node will modify the state.\n",
+ "\n",
+ "Each Edge will choose which node to call next.\n",
+ "\n",
+ "Next Steps:\n",
+ "1. Retrieve Relevant Documents\n",
+ "2. If Relevant Document not found, Go for Supplement Retrieval with Web search(using Tavily API).\n",
+ "3. Query Re-writing to optimize the query for Web search.\n",
+ "\n",
+ "Here is our graph flow:\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "wbZKwW8sqZF1"
+ },
+ "outputs": [],
+ "source": [
+ "import json\n",
+ "import operator\n",
+ "from typing import Annotated, Sequence, TypedDict\n",
+ "\n",
+ "from langchain import hub\n",
+ "from langchain.output_parsers.openai_tools import PydanticToolsParser\n",
+ "from langchain.prompts import PromptTemplate\n",
+ "from langchain.schema import Document\n",
+ "from langchain_community.tools.tavily_search import TavilySearchResults\n",
+ "from langchain_core.messages import BaseMessage, FunctionMessage\n",
+ "from langchain_core.output_parsers import StrOutputParser\n",
+ "from langchain_core.pydantic_v1 import BaseModel, Field\n",
+ "from langchain_core.runnables import RunnablePassthrough\n",
+ "from langchain_core.utils.function_calling import convert_to_openai_tool\n",
+ "\n",
+ "\n",
+ "def retrieve(state):\n",
+ " \"\"\"\n",
+ " Helper function for retrieving documents\n",
+ "\n",
+ " Args:\n",
+ " state (dict): The current graph state\n",
+ "\n",
+ " Returns:\n",
+ " state (dict): New key added to state, documents, that contains retrieved documents\n",
+ " \"\"\"\n",
+ " print(\"*\" * 5, \" RETRIEVE \", \"*\" * 5)\n",
+ " state_dict = state[\"keys\"]\n",
+ " question = state_dict[\"question\"]\n",
+ " documents = retriever.get_relevant_documents(question)\n",
+ " return {\"keys\": {\"documents\": documents, \"question\": question}}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Q3NxXlZKLT6h"
+ },
+ "outputs": [],
+ "source": [
+ "def generate(state):\n",
+ " \"\"\"\n",
+ " Helper function for generating answers\n",
+ "\n",
+ " Args:\n",
+ " state (dict): The current graph state\n",
+ "\n",
+ " Returns:\n",
+ " state (dict): New key added to state, generation, that contains LLM generation\n",
+ " \"\"\"\n",
+ " print(\"*\" * 5, \" GENERATE \", \"*\" * 5)\n",
+ " state_dict = state[\"keys\"]\n",
+ " question = state_dict[\"question\"]\n",
+ " documents = state_dict[\"documents\"]\n",
+ "\n",
+ " # Prompt\n",
+ " prompt = hub.pull(\"rlm/rag-prompt\")\n",
+ "\n",
+ " # LLM\n",
+ " llm = ChatOpenAI(model_name=\"gpt-4-0125-preview\", temperature=0, streaming=True)\n",
+ "\n",
+ " # Nested function for Post-processing retrieved docs\n",
+ " def format_docs(docs):\n",
+ " return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
+ "\n",
+ " # RAG Chain\n",
+ " rag_chain = prompt | llm | StrOutputParser()\n",
+ "\n",
+ " # Run generation\n",
+ " generation = rag_chain.invoke({\"context\": documents, \"question\": question})\n",
+ " return {\n",
+ " \"keys\": {\"documents\": documents, \"question\": question, \"generation\": generation}\n",
+ " }"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "2i0xLjQiLZnb"
+ },
+ "outputs": [],
+ "source": [
+ "def grade_documents(state):\n",
+ " \"\"\"\n",
+ " Determines whether the retrieved documents are relevant to the question.\n",
+ "\n",
+ " Args:\n",
+ " state (dict): The current graph state\n",
+ "\n",
+ " Returns:\n",
+ " state (dict): Updates documents key with relevant documents\n",
+ " \"\"\"\n",
+ "\n",
+ " print(\"*\" * 5, \" DOCS RELEVANCE CHECK\", \"*\" * 5)\n",
+ " state_dict = state[\"keys\"]\n",
+ " question = state_dict[\"question\"]\n",
+ " documents = state_dict[\"documents\"]\n",
+ "\n",
+ " # Data model\n",
+ " class grade(BaseModel):\n",
+ " \"\"\"Binary score for relevance check.\"\"\"\n",
+ "\n",
+ " binary_score: str = Field(description=\"Relevance score 'yes' or 'no'\")\n",
+ "\n",
+ " # LLM\n",
+ " model = ChatOpenAI(temperature=0, model=\"gpt-4-0125-preview\", streaming=True)\n",
+ "\n",
+ " # Tool\n",
+ " grade_tool_oai = convert_to_openai_tool(grade)\n",
+ "\n",
+ " # LLM with tool and enforce invocation\n",
+ " llm_with_tool = model.bind(\n",
+ " tools=[convert_to_openai_tool(grade_tool_oai)],\n",
+ " tool_choice={\"type\": \"function\", \"function\": {\"name\": \"grade\"}},\n",
+ " )\n",
+ "\n",
+ " # Parser\n",
+ " parser_tool = PydanticToolsParser(tools=[grade])\n",
+ "\n",
+ " # Prompt\n",
+ " prompt = PromptTemplate(\n",
+ " template=\"\"\"You are a grader assessing relevance of a retrieved document to a user question. \\n\n",
+ " Here is the retrieved document: \\n\\n {context} \\n\\n\n",
+ " Here is the user question: {question} \\n\n",
+ " If the document contains keyword(s) or semantic meaning related to the user question, grade it as relevant. \\n\n",
+ " Give a binary score 'yes' or 'no' score to indicate whether the document is relevant to the question.\"\"\",\n",
+ " input_variables=[\"context\", \"question\"],\n",
+ " )\n",
+ "\n",
+ " # Chain\n",
+ " chain = prompt | llm_with_tool | parser_tool\n",
+ "\n",
+ " # Score\n",
+ " filtered_docs = []\n",
+ " search = \"No\" # Default do not opt for web search to supplement retrieval\n",
+ " for d in documents:\n",
+ " score = chain.invoke({\"question\": question, \"context\": d.page_content})\n",
+ " grade = score[0].binary_score\n",
+ " if grade == \"yes\":\n",
+ " print(\"*\" * 5, \" RATED DOCUMENT: RELEVANT\", \"*\" * 5)\n",
+ " filtered_docs.append(d)\n",
+ " else:\n",
+ " print(\"*\" * 5, \" RATED DOCUMENT: NOT RELEVANT\", \"*\" * 5)\n",
+ " search = \"Yes\" # Perform web search when documents are not relevant\n",
+ " continue\n",
+ "\n",
+ " return {\n",
+ " \"keys\": {\n",
+ " \"documents\": filtered_docs,\n",
+ " \"question\": question,\n",
+ " \"run_web_search\": search,\n",
+ " }\n",
+ " }"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Q8G1rafvNWjI"
+ },
+ "outputs": [],
+ "source": [
+ "def transform_query(state):\n",
+ " \"\"\"\n",
+ " Helper function for transforming the query to produce a better question.\n",
+ "\n",
+ " Args:\n",
+ " state (dict): The current graph state\n",
+ "\n",
+ " Returns:\n",
+ " state (dict): Updates question key with a re-phrased question\n",
+ " \"\"\"\n",
+ "\n",
+ " print(\"*\" * 5, \"TRANSFORM QUERY\", \"*\" * 5)\n",
+ " state_dict = state[\"keys\"]\n",
+ " question = state_dict[\"question\"]\n",
+ " documents = state_dict[\"documents\"]\n",
+ "\n",
+ " # Create a prompt template with format instructions and the query\n",
+ " prompt = PromptTemplate(\n",
+ " template=\"\"\"You are generating questions that is well optimized for retrieval. \\n\n",
+ " Look at the input and try to reason about the underlying sematic intent / meaning. \\n\n",
+ " Here is the initial question:\n",
+ " \\n --------- \\n\n",
+ " {question}\n",
+ " \\n --------- \\n\n",
+ " Formulate an improved question: \"\"\",\n",
+ " input_variables=[\"question\"],\n",
+ " )\n",
+ "\n",
+ " # Grader model\n",
+ " model = ChatOpenAI(temperature=0, model=\"gpt-4-0125-preview\", streaming=True)\n",
+ "\n",
+ " # Prompt\n",
+ " chain = prompt | model | StrOutputParser()\n",
+ " better_question = chain.invoke({\"question\": question})\n",
+ "\n",
+ " return {\"keys\": {\"documents\": documents, \"question\": better_question}}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LEmdZZlSNdOj"
+ },
+ "outputs": [],
+ "source": [
+ "def web_search(state):\n",
+ " \"\"\"\n",
+ " Helper function to do Web search based on the re-phrased question using Tavily API.\n",
+ "\n",
+ " Args:\n",
+ " state (dict): The current graph state\n",
+ "\n",
+ " Returns:\n",
+ " state (dict): Updates documents key with appended web results\n",
+ " \"\"\"\n",
+ "\n",
+ " print(\"*\" * 5, \" WEB SEARCH \", \"*\" * 5)\n",
+ " state_dict = state[\"keys\"]\n",
+ " question = state_dict[\"question\"]\n",
+ " documents = state_dict[\"documents\"]\n",
+ "\n",
+ " tool = TavilySearchResults()\n",
+ " docs = tool.invoke({\"query\": question})\n",
+ " web_results = \"\\n\".join([d[\"content\"] for d in docs])\n",
+ " web_results = Document(page_content=web_results)\n",
+ " documents.append(web_results)\n",
+ "\n",
+ " return {\"keys\": {\"documents\": documents, \"question\": question}}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "CzGmF_OvNeb_"
+ },
+ "source": [
+ "### Graph Edges"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "HH4S_C95NgOi"
+ },
+ "outputs": [],
+ "source": [
+ "def decide_to_generate(state):\n",
+ " \"\"\"\n",
+ " Helper function to determine whether to generate an answer or re-generate a question for web search.\n",
+ "\n",
+ " Args:\n",
+ " state (dict): The current state of the agent, including all keys.\n",
+ "\n",
+ " Returns:\n",
+ " str: Next node to call\n",
+ " \"\"\"\n",
+ "\n",
+ " print(\"*\" * 5, \" DECIDE TO GENERATE \", \"*\" * 5)\n",
+ " state_dict = state[\"keys\"]\n",
+ " question = state_dict[\"question\"]\n",
+ " filtered_documents = state_dict[\"documents\"]\n",
+ " search = state_dict[\"run_web_search\"]\n",
+ "\n",
+ " if search == \"Yes\":\n",
+ " # All documents have been filtered check_relevance\n",
+ " # We will re-generate a new query\n",
+ " print(\"*\" * 5, \" DECISION: TRANSFORM QUERY and RUN WEB SEARCH \", \"*\" * 5)\n",
+ " return \"transform_query\"\n",
+ " else:\n",
+ " # We have relevant documents, so generate answer\n",
+ " print(\"*\" * 5, \" DECISION: GENERATE \", \"*\" * 5)\n",
+ " return \"generate\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "LT1JY4K5cuHy"
+ },
+ "source": [
+ "### Build Graph\n",
+ "Follow the flow we outlined in the figure above."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "hdOUi1ehqjRD"
+ },
+ "outputs": [],
+ "source": [
+ "import pprint\n",
+ "\n",
+ "from langgraph.graph import END, StateGraph\n",
+ "\n",
+ "workflow = StateGraph(GraphState)\n",
+ "\n",
+ "# Define the nodes\n",
+ "workflow.add_node(\"retrieve\", retrieve) # retrieve docs\n",
+ "workflow.add_node(\"grade_documents\", grade_documents) # grade retrieved docs\n",
+ "workflow.add_node(\"generate\", generate) # generate answers\n",
+ "workflow.add_node(\"transform_query\", transform_query) # transform_query for web search\n",
+ "workflow.add_node(\"web_search\", web_search) # web search\n",
+ "\n",
+ "# Build graph\n",
+ "workflow.set_entry_point(\"retrieve\")\n",
+ "workflow.add_edge(\"retrieve\", \"grade_documents\")\n",
+ "workflow.add_conditional_edges(\n",
+ " \"grade_documents\",\n",
+ " decide_to_generate,\n",
+ " {\n",
+ " \"transform_query\": \"transform_query\",\n",
+ " \"generate\": \"generate\",\n",
+ " },\n",
+ ")\n",
+ "workflow.add_edge(\"transform_query\", \"web_search\")\n",
+ "workflow.add_edge(\"web_search\", \"generate\")\n",
+ "workflow.add_edge(\"generate\", END)\n",
+ "\n",
+ "# Compile\n",
+ "app = workflow.compile()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "sv5vy95qqn7m",
+ "outputId": "9ad28fb7-8013-468e-fb76-fe4630769d5b"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "***** RETRIEVE *****\n",
+ "{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3519055247306824}),\n",
+ " Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.35345977544784546}),\n",
+ " Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nJay Alammar\\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\\n\\n\\nBlog\\nAbout\\n\\n\\n\\n\\n\\n\\nThe Illustrated Transformer\\n\\nDiscussions:\\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\\n\\n\\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\\n\\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3630693554878235}),\n",
+ " Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.37803786993026733})],\n",
+ " 'question': 'How Transformers work?'}\n",
+ "'-------'\n",
+ "***** DOCS RELEVANCE CHECK *****\n",
+ "***** RATED DOCUMENT: RELEVANT *****\n",
+ "***** RATED DOCUMENT: RELEVANT *****\n",
+ "***** RATED DOCUMENT: RELEVANT *****\n",
+ "***** RATED DOCUMENT: RELEVANT *****\n",
+ "{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3519055247306824}),\n",
+ " Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.35345977544784546}),\n",
+ " Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nJay Alammar\\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\\n\\n\\nBlog\\nAbout\\n\\n\\n\\n\\n\\n\\nThe Illustrated Transformer\\n\\nDiscussions:\\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\\n\\n\\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\\n\\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3630693554878235}),\n",
+ " Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.37803786993026733})],\n",
+ " 'question': 'How Transformers work?',\n",
+ " 'run_web_search': 'No'}\n",
+ "'-------'\n",
+ "***** DECIDE TO GENERATE *****\n",
+ "***** DECISION: GENERATE *****\n",
+ "***** GENERATE *****\n",
+ "{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3519055247306824}),\n",
+ " Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.35345977544784546}),\n",
+ " Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nJay Alammar\\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\\n\\n\\nBlog\\nAbout\\n\\n\\n\\n\\n\\n\\nThe Illustrated Transformer\\n\\nDiscussions:\\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\\n\\n\\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\\n\\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3630693554878235}),\n",
+ " Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.37803786993026733})],\n",
+ " 'generation': 'Transformers work by using attention mechanisms to boost the '\n",
+ " 'speed of training deep learning models. They outperform '\n",
+ " 'previous models like the Google Neural Machine Translation '\n",
+ " 'model in specific tasks, mainly due to their ability to '\n",
+ " 'parallelize operations, making them highly efficient for '\n",
+ " 'training on modern hardware. Transformers use self-attention '\n",
+ " 'to weigh the significance of different words in a sentence, '\n",
+ " 'allowing the model to focus on relevant words and ignore less '\n",
+ " 'relevant ones, improving the performance of tasks like '\n",
+ " 'translation and text summarization.',\n",
+ " 'question': 'How Transformers work?'}\n",
+ "'-------'\n",
+ "{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3519055247306824}),\n",
+ " Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.35345977544784546}),\n",
+ " Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nJay Alammar\\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\\n\\n\\nBlog\\nAbout\\n\\n\\n\\n\\n\\n\\nThe Illustrated Transformer\\n\\nDiscussions:\\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\\n\\n\\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\\n\\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3630693554878235}),\n",
+ " Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.37803786993026733})],\n",
+ " 'generation': 'Transformers work by using attention mechanisms to boost the '\n",
+ " 'speed of training deep learning models. They outperform '\n",
+ " 'previous models like the Google Neural Machine Translation '\n",
+ " 'model in specific tasks, mainly due to their ability to '\n",
+ " 'parallelize operations, making them highly efficient for '\n",
+ " 'training on modern hardware. Transformers use self-attention '\n",
+ " 'to weigh the significance of different words in a sentence, '\n",
+ " 'allowing the model to focus on relevant words and ignore less '\n",
+ " 'relevant ones, improving the performance of tasks like '\n",
+ " 'translation and text summarization.',\n",
+ " 'question': 'How Transformers work?'}\n",
+ "'-------'\n",
+ "***** Generated Answer *****\n",
+ "('Transformers work by using attention mechanisms to boost the speed of '\n",
+ " 'training deep learning models. They outperform previous models like the '\n",
+ " 'Google Neural Machine Translation model in specific tasks, mainly due to '\n",
+ " 'their ability to parallelize operations, making them highly efficient for '\n",
+ " 'training on modern hardware. Transformers use self-attention to weigh the '\n",
+ " 'significance of different words in a sentence, allowing the model to focus '\n",
+ " 'on relevant words and ignore less relevant ones, improving the performance '\n",
+ " 'of tasks like translation and text summarization.')\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Run\n",
+ "query_prompt = \"How Transformers work?\"\n",
+ "inputs = {\"keys\": {\"question\": query_prompt}}\n",
+ "for output in app.stream(inputs):\n",
+ " for key, value in output.items():\n",
+ " # Node\n",
+ " # print full state at each node\n",
+ " pprint.pprint(value[\"keys\"], indent=2, width=80, depth=None)\n",
+ " pprint.pprint(\"------------------------\")\n",
+ "\n",
+ "# Final generation\n",
+ "print(\"*\" * 5, \" Generated Answer \", \"*\" * 5)\n",
+ "pprint.pprint(value[\"keys\"][\"generation\"])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "7twOSBPJqwyJ",
+ "outputId": "be4a22b4-a5f9-45df-f063-03265382eb68"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "***** RETRIEVE *****\n",
+ "{ 'documents': [ Document(page_content='Featured in courses at Stanford, Harvard, MIT, Princeton, CMU and others\\nIn the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.', metadata={'vector': [-0.03292850777506828, 0.0013406849466264248, 0.021762628108263016, -0.010678051970899105, 0.0201500803232193, 0.014241919852793217, -0.006924473214894533, -0.016965635120868683, -0.013428870588541031, -0.04143843054771423, 0.03571997955441475, 0.03726477548480034, -0.007520709186792374, 0.0007080307113938034, 0.015868017449975014, -0.006494234316051006, 0.03048936277627945, 0.009031626395881176, -0.008116945624351501, -0.020448198541998863, -0.025692369788885117, 0.009248439222574234, -0.022697634994983673, -0.020082324743270874, -0.009343295358121395, 0.009939531795680523, 0.04967733472585678, -0.03284720331430435, -0.009153584018349648, -0.03181734308600426, 0.015447943471372128, 0.013916700147092342, -0.01604417897760868, -0.028754856437444687, -0.008184699341654778, -0.011565630324184895, 0.011524978093802929, -0.012690350413322449, 0.01742636412382126, -0.015542799606919289, 0.0319257490336895, 0.024567648768424988, 0.008469266816973686, -0.009255214594304562, -0.011342042125761509, 0.006175789516419172, -0.0009985265787690878, -0.04482613503932953, -0.0019665637519210577, 0.015122723765671253, 0.014634894207119942, 0.03905348479747772, -0.02326676994562149, -0.02559751458466053, -0.027101656422019005, 0.0002434914349578321, -0.010081815533339977, 0.007771399803459644, 0.02009587548673153, -0.011111677624285221, 0.01832071878015995, 0.0027745317202061415, -0.008218576200306416, 0.005962363909929991, -0.011545304208993912, -0.015136274509131908, -0.0016371094388887286, 0.03127530962228775, 0.0014186021871864796, 0.010576420463621616, 0.03777970373630524, 0.024635402485728264, 0.0042447964660823345, -0.011965380050241947, 0.035421863198280334, -0.016924982890486717, -0.014743301086127758, -0.00861155055463314, -0.016924982890486717, 0.013550828211009502, -0.0007380965980701149, -0.0036993760149925947, 0.009688841179013252, -0.0007012552814558148, 0.0016218647360801697, -0.0068228417076170444, 0.00879448652267456, -0.008814812637865543, -0.009973408654332161, 0.010833886452019215, -0.0008122027502395213, 0.001707404269836843, 0.01798194646835327, 0.014661995694041252, 0.01265647355467081, 0.014431631192564964, -0.0019835021812468767, 0.030895886942744255, -0.0033165651839226484, -0.01651845872402191, -0.0026559620164334774, -0.001112861675210297, -0.04021885618567467, -0.016247441992163658, -0.019093114882707596, 0.014729749411344528, 0.01596287451684475, -0.006545050069689751, -0.013943801634013653, -0.006392603274434805, -0.0008435390191152692, 0.02023138478398323, 0.0015109172090888023, -0.04290191829204559, -0.0036587235517799854, 0.003594357054680586, 0.009600760415196419, -0.008096619509160519, 0.007961111143231392, -0.01205346081405878, 0.04330844432115555, 0.01519047748297453, 0.005271271802484989, 0.0003809052868746221, 0.007527485024183989, -0.009797248058021069, -0.005406780168414116, -0.0052983732894063, 0.003413114696741104, -0.021207043901085854, -0.02174907736480236, 0.017290854826569557, -0.0016303339507430792, -0.004966378211975098, -0.0177922360599041, 0.014052208513021469, -0.0031996893230825663, -0.01643715240061283, -0.04198046028614044, -0.0023663132451474667, 0.0019835021812468767, -0.006965125445276499, -0.006318073719739914, -0.01296814251691103, -0.0028778568375855684, 0.03409387916326523, 0.005105274263769388, 0.040787987411022186, 0.02352423593401909, 0.022724736481904984, 0.022670533508062363, -0.026952596381306648, -0.008191474713385105, 0.01121330913156271, 0.03206125646829605, 0.016423601657152176, 0.014241919852793217, 0.006734761409461498, -0.007622340694069862, -0.0013991228770464659, 0.00577603979036212, 0.008035640232264996, -0.010251200757920742, -0.014296123757958412, 0.014783953316509724, 0.023456482216715813, 0.005850569345057011, -0.006985451560467482, 0.009438150562345982, -0.02090892568230629, -0.005806529428809881, 0.021369654685258865, -0.04561208188533783, 0.010542543604969978, -0.0034639304503798485, -0.0026119218673557043, 0.03336213529109955, -0.009390722960233688, -0.022494371980428696, -0.01986551284790039, 0.03658723086118698, -0.010481564328074455, 0.012595494277775288, 0.008537020534276962, -0.0008342228247784078, -0.013605031184852123, 0.010522217489778996, -0.013970903120934963, 0.011511427350342274, -0.030922988429665565, 0.009560108184814453, 0.03187154605984688, 0.0065484377555549145, -0.01487880852073431, -0.6348291635513306, -0.008726732805371284, -0.012859735637903214, -0.008963871747255325, 0.015176926739513874, -0.0005263649509288371, 0.0003383472212590277, 0.00016568003047723323, -0.01007504016160965, 0.012568392790853977, -0.0016574356704950333, -0.021112188696861267, -0.011389469727873802, 0.001235665986314416, -0.01910666562616825, -0.02141030691564083, 0.0029930388554930687, -0.03203415498137474, 0.020434647798538208, 0.026342809200286865, -0.03263039141893387, 0.013530501164495945, -0.02386300638318062, 0.005145926494151354, -0.009925981052219868, 0.011484325863420963, 0.009593985043466091, 0.009783697314560413, 0.005173027981072664, 0.012981693260371685, -0.03168183192610741, 0.0036621112376451492, 0.03371445834636688, -0.00020156856044195592, 0.026559622958302498, 0.00534580135717988, 0.005657470319420099, 0.03352474421262741, 0.0042447964660823345, 0.03712926432490349, -0.008719957433640957, -0.020746316760778427, 0.02899877168238163, 0.02197944186627865, 0.026952596381306648, 0.012202519923448563, 0.0065823146142065525, 0.004586955066770315, -0.010217323899269104, -0.02661382593214512, -0.012548066675662994, 0.014201267622411251, 0.01973000355064869, -0.006555213127285242, 0.0006495927809737623, 0.005274659488350153, 0.033172424882650375, -0.0024272918235510588, 0.018252963200211525, -0.01756187155842781, 0.0015651206485927105, 0.01214154064655304, -0.004705524537712336, -0.010854212567210197, -0.018673039972782135, -0.007032879628241062, 0.011904401704668999, 0.026451215147972107, 0.007229366805404425, -0.027752095833420753, 0.004088961984962225, 0.024405039846897125, 0.011050699278712273, 0.00901807565242052, -0.007690094877034426, 0.02023138478398323, 0.018225861713290215, 0.00901807565242052, 0.016681067645549774, 0.032278068363666534, 0.021396756172180176, -0.02172197587788105, 0.015217579901218414, -0.02680353820323944, -0.005400004331022501, -0.0010137712815776467, -0.015014316886663437, -0.006477295886725187, -0.007107409182935953, -0.02256212756037712, 0.017074041068553925, 0.011220084503293037, -0.030164143070578575, -0.05745550990104675, -0.012270273640751839, 0.010108917020261288, 0.0006953268311917782, 0.009600760415196419, 0.017521219328045845, -0.00421430729329586, 0.0015625798841938376, -0.020014571025967598, -0.0041702669113874435, 0.010915190912783146, 0.01758897304534912, -0.0001675856183283031, -0.008774160407483578, 0.029188483953475952, 0.027277816087007523, -0.01627454347908497, -0.01680302619934082, 0.003396176267415285, -0.014255470596253872, 0.019323479384183884, 0.00320307700894773, -0.027521731331944466, 0.017656726762652397, -0.012893612496554852, -0.010047937743365765, -0.018198760226368904, 0.014282572083175182, -0.01025797612965107, 0.006606028415262699, -0.0012458291603252292, 0.009648188948631287, 0.004088961984962225, 0.013381442986428738, -0.02641056291759014, -0.011917952448129654, 0.015637654811143875, 0.010630623437464237, -0.001188238151371479, 0.020380442962050438, -0.011599508114159107, 0.00985145103186369, 0.019242174923419952, -0.0037739055696874857, -0.003831496462225914, -0.00014493032358586788, -0.017697380855679512, -0.008157597854733467, 0.014309674501419067, 0.01677592284977436, 0.026112444698810577, -0.05485375225543976, 0.008340533822774887, -0.011335266754031181, 0.0035096644423902035, -0.015881570056080818, -0.013984453864395618, -0.0005090029444545507, -0.012351579032838345, -0.027630137279629707, 0.007330997847020626, -0.01967580057680607, -0.018794996663928032, 0.004048309754580259, -0.04333554580807686, -0.020922476425766945, -0.043796271085739136, 0.027752095833420753, 0.0458017960190773, -0.029405295848846436, 0.023930760100483894, -0.013449196703732014, -0.009160359390079975, -0.00341819622553885, 0.043687865138053894, -0.025502657517790794, -0.030299650505185127, 0.021207043901085854, -0.04479903355240822, 0.01815810799598694, 0.002185070887207985, -0.003845047438517213, 0.001308501698076725, -0.007662992924451828, -0.01417416613548994, -0.015868017449975014, -0.014634894207119942, -0.009309418499469757, 0.01981130987405777, -0.014987215399742126, -0.022670533508062363, 0.022467270493507385, 0.01174856722354889, 0.039161890745162964, 0.00645019393414259, -0.025258744135499, 0.010860987938940525, -0.0018124232301488519, -0.01454003807157278, -0.0007321680895984173, 0.016504907980561256, 0.00703965499997139, 0.008083067834377289, 0.020705662667751312, -0.01573251001536846, 0.0006402765866369009, 0.022521473467350006, -0.004471772816032171, -0.0008041569381020963, 0.012216070666909218, -0.02295510098338127, 0.009729493409395218, 0.00181072938721627, 0.0012178805191069841, -0.029405295848846436, 0.0005958129186183214, -0.004512425512075424, 0.01947253756225109, -0.01294104102998972, -0.01894405670464039, -0.023117709904909134, -0.019960368052124977, 0.03870116174221039, -0.005247557535767555, 0.029974430799484253, -0.01930992864072323, 0.0007728206692263484, 0.011125229299068451, -0.001856463379226625, -0.009499129839241505, -0.008719957433640957, 0.005809917114675045, 0.02397141233086586, 0.024188226088881493, 0.011328491382300854, -0.0037806809414178133, -0.0163558479398489, 0.00425834720954299, 0.016152584925293922, 0.012920713983476162, 0.028104417026042938, 0.009526231326162815, -0.003353829961270094, 0.013571154326200485, -0.01333401445299387, 0.018022600561380386, -0.008042415603995323, 0.00514253880828619, 0.008550572209060192, -0.014648444950580597, -0.024066269397735596, 0.024581199511885643, 0.010373158380389214, 0.03444620221853256, -0.014838156290352345, -0.006927860900759697, -0.00858444906771183, -0.015271782875061035, -0.005074784625321627, 0.004915562458336353, 0.008137271739542484, 0.004688586108386517, 0.0014228367945179343, 0.004454834386706352, -0.006667007226496935, 0.02216915227472782, 0.028429636731743813, 0.010068264789879322, 0.009715942665934563, 0.02155936509370804, -0.011545304208993912, 0.017494117841124535, 0.009661739692091942, 0.013638908043503761, -0.011735016480088234, -0.009255214594304562, -0.007696870248764753, -0.0037976193707436323, -0.01664041541516781, 0.004908787086606026, -0.0489455871284008, 0.008760609664022923, 0.022128500044345856, -0.011538528837263584, 0.034663014113903046, -0.010332505218684673, 0.044419609010219574, -0.008862241171300411, -0.05244170501828194, 0.004454834386706352, -0.0031082211062312126, -0.01981130987405777, -0.02040754444897175, -0.03208835795521736, -0.00602673040702939, -0.008408288471400738, 0.003250504843890667, -0.01664041541516781, 0.004400630947202444, -0.0019987470004707575, 0.00036036729579791427, -0.015691857784986496, -0.0027271038852632046, 0.022304661571979523, -0.005945425480604172, 0.012703901156783104, -0.028565144166350365, -0.0015532636316493154, 0.006958350073546171, -0.03450040519237518, -0.006999002769589424, 0.02206074632704258, -0.020814070478081703, -0.018469776958227158, -0.010501890443265438, -0.008191474713385105, -0.020475300028920174, 0.026735782623291016, -0.004082186613231897, 0.010474788956344128, -0.03360605239868164, 0.014052208513021469, -0.006826229393482208, 0.014716198667883873, 0.009986959397792816, 0.0251232348382473, 0.017887091264128685, -0.0022087846882641315, -0.01823941245675087, 0.008537020534276962, 0.022697634994983673, 0.07252402603626251, 0.01792774349451065, -0.033118221908807755, 0.014092860743403435, -0.03937870264053345, -0.003485950408503413, -0.020326239988207817, -0.027467528358101845, 0.009966633282601833, -0.00817792396992445, -0.00762911606580019, -0.013347565196454525, 0.010915190912783146, -0.012446435168385506, 0.03303691744804382, 0.026491869240999222, -0.005610042251646519, -0.015786712989211082, -0.005349189043045044, -0.016789475455880165, 0.00838796142488718, -0.020272037014365196, 0.029296889901161194, 0.008869016543030739, 0.03433779627084732, -0.019242174923419952, 0.027318468317389488, 0.027860501781105995, 0.016504907980561256, -0.01361180655658245, -0.005427106283605099, -0.01146399974822998, -0.012690350413322449, -0.0026102280244231224, -0.026207301765680313, 0.04217017441987991, -0.0034673181362450123, 0.01700628735125065, 0.00037455331766977906, 0.006284196395426989, 0.004024595487862825, 0.03303691744804382, 0.009953082539141178, -0.011443673633038998, -0.018103905022144318, -0.029947329312562943, 0.02624795399606228, 0.023673294112086296, -0.021884584799408913, -0.024581199511885643, 0.0012170335976406932, 0.0018869527848437428, -0.04433830454945564, 0.005498248152434826, 0.013638908043503761, -0.003618071088567376, 0.003997494000941515, 0.010786457918584347, -0.010752581059932709, -0.006670394912362099, -0.020475300028920174, -0.029974430799484253, -0.0003730712051037699, 0.00017213785031344742, 0.012913938611745834, -0.0011425040429458022, -0.000349357258528471, -0.018117455765604973, -0.01840202324092388, 0.01907956413924694, 0.006077545695006847, -0.027833400294184685, -0.05170996114611626, -0.005054458510130644, 0.02661382593214512, 0.006978676188737154, 0.02989312633872032, -0.008801261894404888, -0.02054305374622345, 0.021992992609739304, -0.008462491445243359, -0.028429636731743813, 0.011782444082200527, -0.02216915227472782, -0.008814812637865543, -0.002630554372444749, -0.015000766143202782, -0.00042049912735819817, -0.013191730715334415, 0.0038247210904955864, -0.0028609184082597494, 0.0011179432040080428, 0.01355760358273983, -0.001084913033992052, 0.0013881127815693617, 0.02967631258070469, 0.024906421080231667, 0.02174907736480236, 0.01986551284790039, -0.033849965780973434, 0.006304522510617971, -0.025245191529393196, -0.005610042251646519, -0.02562461607158184, 0.023388726636767387, 0.005156089551746845, 0.025502657517790794, 0.011992481537163258, 0.017805786803364754, -0.01683012768626213, 0.019608046859502792, -0.0036756619811058044, -0.010298628360033035, -0.0033894008956849575, 0.009309418499469757, 0.0027813073247671127, 0.008320207707583904, 0.008841914124786854, -0.008774160407483578, -0.029351092875003815, -0.0160983819514513, -0.027020350098609924, 0.03417518734931946, 0.013198506087064743, -0.003064180724322796, 0.001883565098978579, -0.00045310580753721297, -0.004807156044989824, -0.0189711581915617, 0.021654222160577774, -0.008164373226463795, 0.03347054123878479, 0.010671276599168777, 0.0025915957521647215, -0.029567906633019447, -0.004790217150002718, -0.006887208204716444, -0.009126481600105762, -0.012886837124824524, -0.01683012768626213, 0.004888460971415043, -0.008665753528475761, -0.019770655781030655, -0.013571154326200485, 0.02756238356232643, -0.019093114882707596, -0.023090608417987823, 0.01012924313545227, 0.017182448878884315, 0.02829412929713726, 0.0006085167988203466, 0.00991920568048954, -0.037752602249383926, -0.008001763373613358, 0.0077104209922254086, -0.01790064200758934, 0.003231872571632266, -0.004139777738600969, 0.015095622278749943, 0.02127479761838913, 0.040842194110155106, 0.0018988096853718162, -0.002451005857437849, 0.041411325335502625, -0.0251232348382473, -0.00038386951200664043, 0.014661995694041252, 0.019513191655278206, -0.00783915352076292, -0.003692600643262267, 0.015895120799541473, 0.00600640382617712, 0.01080678403377533, 0.010325729846954346, 0.016084831207990646, 0.001335603417828679, 0.012039910070598125, -0.003651948180049658, -0.010027611628174782, -0.02122059464454651, -0.016789475455880165, 0.021627120673656464, -0.008198250085115433, -0.015041418373584747, -0.02470315806567669, 0.005481309723109007, 0.028077315539121628, 0.02562461607158184, 0.02981182187795639, 0.017467016354203224, 0.016789475455880165, 0.0022900898475199938, -0.006351950578391552, 0.012778431177139282, 0.034663014113903046, -0.026573173701763153, -0.004343039821833372, -0.036993756890296936, -0.005193354561924934, 0.03723767399787903, 0.00972271803766489, 0.004637770354747772, 0.008747058920562267, -0.0035706430207937956, -0.006541662383824587, 0.015868017449975014, -0.007351323962211609, -0.01800904981791973, -0.015488595701754093, -0.013876047916710377, -0.008625101298093796, -0.013279811479151249, -0.018510429188609123, -0.007303896360099316, 0.00802208948880434, -0.003062486881390214, -0.009912430308759212, 0.007873030379414558, -0.028781957924365997, -0.02242661826312542, 0.036126505583524704, 0.016992736607789993, 0.046750351786613464, 0.013002019375562668, 0.030977191403508186, 0.01756187155842781, 0.002720328513532877, -0.0052340067923069, -0.008618325926363468, -0.0018361371476203203, 0.02323966845870018, 0.008076292462646961, 0.0008007692522369325, -0.004105900414288044, -0.031031396239995956, 0.00647052051499486, 0.010298628360033035, -0.00920101162046194, -0.038457248359918594, 0.012541291303932667, 0.013523725792765617, 0.007520709186792374, -0.013774416409432888, -0.027657238766551018, 0.0010451074922457337, 0.010705153457820415, -0.01528533361852169, 0.02402561530470848, 0.011179432272911072, -0.015542799606919289, -0.012527740560472012, 0.0027237161993980408, 0.00455307774245739, 0.010427361354231834, -0.011694363318383694, 0.007649442180991173, 0.033904168754816055, 0.0011433509644120932, 0.0037908439990133047, -0.004698749165982008, 0.0033741560764610767, 0.01448583509773016, -0.02164067141711712, 0.009871777147054672, 0.00895032100379467, -0.006951574701815844, 0.008110170252621174, -0.0010501891374588013, -0.016762372106313705, 0.03214256092905998, -0.037698399275541306, -0.015515698119997978, 0.01714179664850235, -0.015122723765671253, 0.01233802828937769, -0.03902638331055641, -0.0028592245653271675, 0.0018158109160140157, -0.01028507761657238, -0.006853331346064806, 0.017182448878884315, 0.020732766017317772, -0.03951421007514, 0.0057794274762272835, 0.020583705976605415, -0.011626609601080418, -0.013693111948668957, 0.005034132394939661, 0.014214818365871906, -0.03336213529109955, -0.01445873361080885, 0.010833886452019215, -0.0013415318680927157, 0.02680353820323944, -0.014323225244879723, -0.009892103262245655, -0.017399262636899948, 0.02706100232899189, -0.009973408654332161, -0.009343295358121395, 0.015800263732671738, 0.0016709864139556885, -0.010488339699804783, 0.02248082123696804, -0.004525976255536079, -0.017304405570030212, 0.007595238741487265, -0.0223724152892828, -0.019946817308664322, 0.004519200883805752, -0.004309162963181734, 0.0036756619811058044, 0.010515442118048668, 0.0035299905575811863, -0.01598997600376606, 0.005423718597739935, -0.008936770260334015, -0.009451702237129211, -0.00014302475028671324, -0.023564888164401054, -0.007378425914794207, 0.006277421023696661, 0.0329827144742012, -0.030841683968901634, 0.006301134824752808, -0.027941806241869926, -0.010833886452019215, -0.014811054803431034, -0.0371563658118248, 0.002569575561210513, -0.01756187155842781, 0.012399007566273212, -0.0057455506175756454, -0.004000881686806679, 0.022250458598136902, 0.004725851118564606, -0.020475300028920174, 0.01100327167659998, 0.0016642110422253609, 0.0018801772966980934, -0.008198250085115433, 0.010996496304869652, 0.025502657517790794, 0.003485950408503413, -0.03740028291940689, -0.007764624431729317, -0.00018928811186924577, 0.005379678215831518, -0.027657238766551018, -0.021166391670703888, -0.010976170189678669, 0.0282670259475708, 0.007053205743432045, -0.018849199637770653, -0.009844675660133362, -0.018225861713290215, -0.027914704754948616, -0.012852960266172886, -0.013930250890552998, 0.016342297196388245, 0.00320307700894773, 0.01840202324092388, 0.006094484589993954, 0.013117201626300812, 0.0076562175527215, 0.009966633282601833, 0.0057794274762272835, 0.029947329312562943, -0.025895632803440094, -0.009519455954432487, -0.02164067141711712, 0.003367380704730749, -0.046967167407274246, -0.011159106157720089, 0.004475160501897335, -0.023591989651322365, 0.0032386479433625937, 0.019540293142199516, 0.008753834292292595, -0.0035367661621421576, -0.0019716452807188034, 0.01641005091369152, -0.0019360744627192616, 0.019093114882707596, 0.03216966241598129, -0.02697969786822796, -0.0023612314835190773, -0.005471146199852228, -0.02174907736480236, -0.0097498195245862, 0.0050442954525351524, 0.009878552518785, 0.00851669441908598, 0.0029252846725285053, -0.02197944186627865, 0.013930250890552998, 0.019635148346424103, 0.0013779497239738703, 0.017074041068553925, 0.0033639930188655853, -0.0023544561117887497, 0.014960113912820816, 0.01761607453227043, -0.019187970086932182, -0.009966633282601833, 0.014743301086127758, -0.013821844011545181, -0.024486344307661057, 0.026058241724967957, -0.028890365734696388, 0.0019835021812468767, -0.005030744709074497, -0.002498433692380786, -0.008686079643666744, -0.005257721059024334, -0.0006343480781652033, 0.0025390861555933952, -0.026789987459778786, 0.02661382593214512, -0.002362925559282303, -0.03815912827849388, 0.014282572083175182, -0.005874283611774445, 0.018307168036699295, -0.017101144418120384, -0.005677796434611082, 0.009411049075424671, -0.013523725792765617, -0.012019583024084568, 0.00703965499997139, -0.009614312089979649, -0.024039166048169136, -0.002821959787979722, 0.013747314922511578, -0.011680812574923038, -0.009783697314560413, 0.21106769144535065, -0.020001020282506943, -0.0022172541357576847, 0.010278302244842052, 0.011328491382300854, 0.010935517027974129, 0.01800904981791973, 0.016369398683309555, -0.025583963841199875, -0.011809545569121838, 0.03504243865609169, -0.004942663945257664, -0.04249539226293564, -0.006019955035299063, 0.01482460554689169, -0.027277816087007523, -0.03948710858821869, -0.0382133312523365, -0.005925098899751902, -0.004536139313131571, 0.014689097180962563, -0.008970647118985653, -0.004681810736656189, -0.03807782381772995, 0.008869016543030739, 0.0024848829489201307, 0.0009883634047582746, -0.007005778141319752, 0.021491611376404762, 0.024188226088881493, -0.033091120421886444, 0.007168387994170189, 0.012249947525560856, 0.016870779916644096, -0.021030884236097336, -0.00913325697183609, 0.01487880852073431, -0.009275540709495544, 0.011362368240952492, 0.006104647647589445, 0.006873657461255789, 0.007114184554666281, 0.009363621473312378, -0.00199197162874043, 0.00023205792240332812, 0.018483327701687813, -0.02389010787010193, 0.0050612338818609715, -0.004712299909442663, 0.005139151122421026, -0.025746572762727737, 0.0018429126357659698, 0.007568137254565954, 0.036912452429533005, 0.0011170962825417519, 0.019431885331869125, -0.00288801989518106, 0.02012297883629799, 0.03355184569954872, 0.014241919852793217, -0.012961367145180702, 0.011897626332938671, -0.011077800765633583, 0.025611065328121185, 0.014269021339714527, 0.003926352132111788, -0.01815810799598694, 0.035503167659044266, 0.010251200757920742, -0.011457224376499653, -0.011497876606881618, -0.01407930999994278, -0.022182703018188477, -0.008001763373613358, -0.03788811340928078, -0.01666751690208912, 0.04111320897936821, 0.015122723765671253, 0.016586212441325188, 0.026789987459778786, -0.005379678215831518, -0.025041930377483368, 0.007920457981526852, -0.009600760415196419, 0.00966851506382227, -0.028483839705586433, 0.010461238212883472, -0.0030506299808621407, -0.0019208298763260245, -0.004634382668882608, 0.018903404474258423, -0.026261504739522934, -0.010041162371635437, -0.0014507854357361794, -0.02352423593401909, -0.0066737825982272625, 0.033795762807130814, 0.010854212567210197, 0.013483073562383652, -0.013144303113222122, -0.026871291920542717, 0.031654730439186096, 0.032359376549720764, 0.0005687112570740283, -0.0035706430207937956, 0.014431631192564964, 0.014865257777273655, 0.01155207958072424, -0.012094113044440746, -0.003912801388651133, -0.0012678492348641157, -0.050110962241888046, 0.011145555414259434, 0.0017683830810710788, -0.012385456822812557, 0.00951268058270216, -0.020421097055077553, -0.0030269159469753504, 0.0232938714325428, -0.015746060758829117, -0.00011602896120166406, -0.025773674249649048, -0.009180685505270958, 0.006802515592426062, 0.00045818736543878913, -0.016139034181833267, -0.016992736607789993, -0.006440030876547098, -0.021288348361849785, 0.014011556282639503, 0.002318885177373886, -0.04127581790089607, 0.021207043901085854, 0.001632874715141952, -0.014363877475261688, -0.023768151178956032, 7.209463365143165e-05, 0.008035640232264996, 0.012622595764696598, 0.01737215928733349, -0.009363621473312378, -0.007778175175189972, -0.005311924032866955, -0.0030658745672553778, 0.0012000950518995523, -0.02441859059035778, 0.008462491445243359, 0.02124769613146782, 0.013354340568184853, -0.012161866761744022, -0.014634894207119942, 0.023470032960176468, 0.005352576728910208, -0.026234403252601624, 0.022629881277680397, -0.01716889813542366, -0.021600017324090004, -0.03290140628814697, 0.004224470350891352, -0.014675546437501907, -0.03607230260968208, -0.010698378086090088, 0.017629625275731087, -0.0007567289867438376, -0.01669461838901043, -0.0003938209265470505, -0.17236651480197906, -0.0044683851301670074, 0.021627120673656464, -0.025583963841199875, 0.02315836399793625, 0.0035808063112199306, 0.01941833458840847, -0.00407541124150157, -0.022616330534219742, -0.005227231420576572, 0.020529503002762794, -0.022778939455747604, -0.008652202785015106, -0.01865948922932148, -0.0066568441689014435, 0.006971900817006826, -0.014770402573049068, -0.004841032903641462, 0.012609045021235943, 0.008435389958322048, 0.03777970373630524, -0.036234911531209946, 0.007635891437530518, 0.008157597854733467, -0.00817792396992445, 0.0194996390491724, 0.0049867043271660805, 0.03496113419532776, 0.02829412929713726, -0.028619349002838135, -0.0027440425474196672, -0.00861155055463314, 0.04271220788359642, -0.014580690301954746, 0.006463745143264532, 0.013340789824724197, 0.012697125785052776, -0.03490693122148514, -0.025001278147101402, 0.01907956413924694, 0.026261504739522934, -0.004993479698896408, 0.002425597980618477, 0.002478107577189803, -0.019743554294109344, -0.012974917888641357, 0.028890365734696388, -0.013889598660171032, 0.021871034055948257, -0.024147573858499527, 0.027616586536169052, -0.009004524908959866, -0.008137271739542484, 0.006785577163100243, -0.007547811139374971, 0.007195489481091499, 0.004942663945257664, 0.0052340067923069, 0.0011179432040080428, -0.01855108141899109, -0.019377682358026505, 0.0009375478839501739, 0.023849455639719963, 0.0014592546503990889, 0.001086606876924634, -0.011531753465533257, 0.01261582039296627, 0.023171914741396904, -0.031763140112161636, 0.010711928829550743, -0.01632874645292759, 0.025190988555550575, -0.032955609261989594, -0.02250792272388935, 0.00026868749409914017, 0.026993248611688614, -0.03813202679157257, 0.0041872053407132626, 0.003340278984978795, 0.0011475856881588697, -0.012852960266172886, 0.03208835795521736, -0.014418080449104309, 0.014038657769560814, -0.005945425480604172, -0.010610297322273254, -0.003485950408503413, 0.0177922360599041, -0.04431120306253433, -0.020028121769428253, 0.019038911908864975, -0.021112188696861267, -0.002556024817749858, 0.006297747138887644, 0.01519047748297453, 0.021843932569026947, 0.022291110828518867, -0.029567906633019447, 0.02441859059035778, -0.003172587603330612, -0.008462491445243359, 0.013930250890552998, -0.0019462376367300749, -0.007141286041587591, 0.01573251001536846, 0.027237163856625557, -0.01781933754682541, -0.001646425575017929, 0.02978471852838993, 0.005454207770526409, -0.01981130987405777, -0.021464509889483452, 0.01108457613736391, 0.0048207067884504795, -0.0025899019092321396, 0.02028558775782585, -0.0035401538480073214, 0.0037332531064748764, 0.022684084251523018, 0.014838156290352345, 0.059135813266038895, 0.0012509106891229749, -0.012317701242864132, 0.0027830011676996946, -0.01722310110926628, -0.04263089969754219, -0.10585907101631165, -0.020448198541998863, 0.011389469727873802, 0.015840915963053703, 0.00030595227144658566, 0.008679304271936417, -0.014431631192564964, 0.019187970086932182, -0.027237163856625557, 0.028754856437444687, -0.03436489775776863, -0.02528584562242031, 0.024228878319263458, -0.005403392016887665, -0.00291173392906785, -0.019066013395786285, 0.021179942414164543, -0.0011941666016355157, -0.024350836873054504, 0.03954131156206131, -0.008699631318449974, 0.011375918984413147, 0.017385710030794144, -0.020339790731668472, -0.010678051970899105, -0.0032589740585535765, -0.024987727403640747, 0.011897626332938671, 0.01077968254685402, -0.01700628735125065, 0.012243172153830528, -0.024323733523488045, 0.012886837124824524, -0.021030884236097336, 0.02177617885172367, 0.002232498722150922, -0.01567830704152584, -0.0067483121529221535, 0.02028558775782585, -0.018361371010541916, -0.0071006338112056255, -0.00858444906771183, 0.0020851334556937218, 0.010745805688202381, -0.016206789761781693, -0.010488339699804783, -0.019960368052124977, 0.020502401515841484, 0.01139624509960413, -0.0260446909815073, -0.02310415916144848, 0.014404529705643654, -0.03677694499492645, 0.010603521950542927, 0.025773674249649048, 0.0008782630320638418, 0.007466506212949753, -0.011660486459732056, -0.006077545695006847, -0.0021037659607827663, 0.011599508114159107, -0.011050699278712273, -0.017209550365805626, 0.039893634617328644, 0.027332019060850143, -0.009627862833440304, -0.011348817497491837, -0.02567881904542446, -0.0015981508186087012, -0.027887603268027306, -0.0038755368441343307, 0.011809545569121838, 0.005108661949634552, 0.0072090402245521545, -0.033226627856492996, 0.004549690056592226, -0.01745346561074257, -0.012683575041592121, 0.0005555839161388576, -0.0030658745672553778, -0.016789475455880165, -0.01711469516158104, -0.006111423019319773, -0.01795484498143196, -0.011843422427773476, -0.006094484589993954, -0.008753834292292595, -0.03661433607339859, 0.00703965499997139, -0.03704795986413956, 0.002288396004587412, 0.04284771531820297, 0.007107409182935953, 0.0003567678795661777, 0.008266004733741283, -0.003475787350907922, -0.009004524908959866, -0.008990973234176636, -0.003946678247302771, 0.03669564053416252, -0.007723971735686064, -0.011301389895379543, -0.06200858950614929, 0.006382439751178026, -0.008774160407483578, -0.007757849059998989, -0.0014888971345499158, 0.002991345012560487, 0.013577929697930813, 0.005677796434611082, -0.013998005539178848, -0.011348817497491837, -0.023876557126641273, -0.010088590905070305, 0.000378999684471637, -0.0005305995582602918, -0.02905297465622425, -0.022440169006586075, 0.022209804505109787, -0.016870779916644096, 0.01090164016932249, 0.012195744551718235, 0.011972155421972275, 0.006138524506241083, 0.02174907736480236, 0.029513703659176826, -0.014404529705643654, -0.030814582481980324, 0.01688433066010475, 0.012006032280623913, -0.0009527925867587328, -0.0009477109997533262, 0.0046547092497348785, -0.007574912626296282, 0.0057455506175756454, 0.022209804505109787, 0.0014787339605391026, 0.0054474323987960815, -0.0038721489254385233, 0.019621597602963448, 0.013930250890552998, 0.031139802187681198, -0.01936413161456585, -0.05092401057481766, 0.010928741656243801, -0.010088590905070305, -0.01601707749068737, -0.009898878633975983, -0.0009976796573027968, 0.0018666265532374382, 0.013760865665972233, 0.007642666809260845, 0.006511172745376825, 0.01832071878015995, -0.0013457664754241705, -0.0198926143348217, -0.015122723765671253, 0.027697892859578133, 0.01567830704152584, -0.007805276662111282, -0.006453581620007753, -0.0012873285450041294, 0.03653302788734436, 0.004962990526109934, 0.005203517619520426, -0.0012653084704652429, 0.012439659796655178, 0.003784068627282977, -0.025421353057026863, -0.0058370186015963554, 0.005582940764725208, -0.03482562676072121, 0.007391976658254862, -0.009844675660133362, 0.02588208205997944, 0.011978930793702602, 0.024540547281503677, 0.005738775245845318, -0.014905910938978195, -0.011518202722072601, -0.030922988429665565, 0.02593628503382206, 0.013571154326200485, 0.009268765337765217, -0.010190221481025219, 0.01876789517700672, 0.020949577912688255, 0.012697125785052776, -0.006460356991738081, -0.0031200782395899296, -0.01355760358273983, 0.0076562175527215, 0.013666009530425072, 0.012127989903092384, 0.013327239081263542, -0.003397870110347867, -0.001734505989588797, 0.016477804630994797, -0.0002483612624928355, 0.0023392115253955126, 0.013273036107420921, 0.002811796497553587, 0.020759867504239082, -0.0011340348282828927, -0.015312435105443, -0.01790064200758934, -0.04189915582537651, 0.0008054273203015327, -0.030055735260248184, -0.01933703012764454, 0.0008105089073069394, 0.0198926143348217, 0.01745346561074257, -0.015691857784986496, 0.005359352100640535, -0.0006449346547015011, -0.014390978962182999, 0.02402561530470848, 3.385059972060844e-05, -0.009404273703694344, -0.029323991388082504, -0.004238021094352007, 0.03431069478392601, 0.010881314054131508, 0.029378194361925125, 0.005942037794739008, 0.036858249455690384, 0.01863238774240017, 0.008259229362010956, -0.024919971823692322, 0.029703414067626, 0.01834782026708126, -0.007520709186792374, 0.007161612622439861, 0.009682065807282925, -0.013469522818922997, -0.01090164016932249, -0.007114184554666281, -0.008618325926363468, 0.02284669503569603, 0.0001381549081997946, 0.08320207893848419, 0.0175347700715065, -0.010420585982501507, 0.0017175674438476562, 0.004353203345090151, 0.0007808664813637733, 0.01986551284790039, 0.007859479635953903, -0.021112188696861267, 0.014241919852793217, -0.00600640382617712, 0.012609045021235943, -0.029351092875003815, 0.004326101392507553, -0.003574030939489603, 0.017317956313490868, -0.006849943660199642, 0.007466506212949753, -0.015054970048367977, -0.016206789761781693, 0.024947073310613632, -0.02483866736292839, 0.021532263606786728, -0.0011077801464125514, -0.013970903120934963, -0.010854212567210197, 0.0201500803232193, 0.016545560210943222, 0.0054474323987960815, -0.027413325384259224, 0.01907956413924694, 0.006944799330085516, -0.04507005214691162, -0.02475736290216446, 0.0011374225141480565, 3.705304334289394e-05, -0.0032860757783055305, -0.008381186053156853, 0.02973051555454731, -0.0016108546406030655, 0.000994291971437633, 0.03132951259613037, -0.006575539242476225, -0.027860501781105995, -0.015271782875061035, -0.001975032966583967, -0.0024747196584939957, -0.03211545944213867, -0.005552451126277447], '_distance': 0.3664601147174835}),\n",
+ " Document(page_content='The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nJay Alammar\\nVisualizing machine learning one concept at a time.@JayAlammar on Twitter. YouTube Channel\\n\\n\\nBlog\\nAbout\\n\\n\\n\\n\\n\\n\\nThe Illustrated Transformer\\n\\nDiscussions:\\nHacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)\\n\\n\\nTranslations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish 2, Vietnamese\\n\\nWatch: MIT’s Deep Learning State of the Art lecture referencing this post', metadata={'vector': [-0.01995760016143322, -0.01293110754340887, 0.02453695610165596, -0.007925305515527725, 0.016309859231114388, -0.010300273075699806, -0.006226088851690292, -0.002630834234878421, -0.02243753708899021, -0.032147347927093506, 0.023093605414032936, 0.026820074766874313, -0.002322481945157051, 0.004969717934727669, 0.012708044610917568, 0.009886950254440308, 0.02301487885415554, 0.0023405239917337894, -0.007971230894327164, -0.016651015728712082, -0.015076451003551483, 0.02242441661655903, 0.00028661987744271755, -0.03180619329214096, -0.0037658323999494314, 0.010201863013207912, 0.02721371501684189, -0.032357290387153625, 0.022043896839022636, -0.016782227903604507, 0.028027240186929703, 0.0006700098165310919, -0.010306834243237972, -0.009486748836934566, -0.008135247975587845, 0.0006921521271578968, 0.020285634323954582, -0.007361087016761303, 0.027502385899424553, -0.025101175531744957, 0.031570009887218475, 0.002647235756739974, 0.0021814273204654455, -0.0353752076625824, -0.002237193053588271, 0.030310358852148056, 0.0003009713545907289, -0.021033551543951035, -0.008496085181832314, 0.04450767859816551, 0.021505920216441154, 0.025770364329218864, -0.006980567239224911, -0.028630822896957397, -0.02642643265426159, 0.019918235018849373, 0.00011686217476380989, 0.01732020452618599, 0.013154170475900173, 0.004474386107176542, 0.004326771013438702, -0.0005761100328527391, -0.01124501135200262, -0.017188990488648415, -0.001525358995422721, -0.02207013964653015, -0.020967945456504822, 0.019577080383896828, 0.0213222224265337, 8.744161459617317e-05, 0.041411034762859344, 0.026767589151859283, -0.01171081978827715, -0.032934632152318954, 0.039783984422683716, -0.0151945436373353, 0.0047335331328213215, -0.004129950422793627, -0.01852736994624138, 0.00655740313231945, 0.010044406168162823, 0.010569261386990547, 0.00771536398679018, 0.018986618146300316, 0.007144584320485592, 0.004782738164067268, -0.005258387885987759, 0.0020764563232660294, -0.018028758466243744, -0.015627548098564148, 0.00010122929961653426, 0.0034902836196124554, 0.016677258536219597, 0.010333077050745487, -0.00450390949845314, 0.020443089306354523, -0.002519302535802126, 0.019078467041254044, 0.017031535506248474, -0.02179459109902382, -0.003933129832148552, -0.0039036066737025976, -0.009440823458135128, -0.014525353908538818, -0.023237941786646843, 0.005724196322262287, 0.007275797892361879, 0.019498351961374283, 0.011330300942063332, 0.0019173597684130073, 7.790811650920659e-05, 0.02254250831902027, -0.010713595896959305, -0.03954780101776123, -0.0042775655165314674, -0.031989894807338715, 0.009342413395643234, -0.00810900516808033, 0.029680533334612846, -0.021282857283949852, 0.03571636229753494, 0.03422052413225174, -0.005442087072879076, 0.0020584145095199347, 0.017556389793753624, 0.016126159578561783, -0.005465049296617508, -0.004484227392822504, -0.028604580089449883, -0.0044678254052996635, -0.011015388183295727, 0.008384553715586662, -0.0015712836757302284, -0.0023634862154722214, -0.025363603606820107, 0.008318946696817875, -0.0072561162523925304, 0.012675240635871887, -0.012944228947162628, -0.020246269181370735, -0.0024733776226639748, -0.0027341649401932955, 0.0027718888595700264, -0.011730502359569073, -0.01111379824578762, 0.028027240186929703, -0.009598280303180218, 0.041935890913009644, -0.007813774049282074, -0.0005416664644144475, 0.004461264703422785, -0.01995760016143322, -0.001276052906177938, 0.012806454673409462, 0.014525353908538818, 0.016296738758683205, 0.0022306323517113924, 0.029523076489567757, -0.00564546836540103, 0.015299513936042786, -0.0012506303610280156, 0.013088563457131386, -0.020705517381429672, -0.01822557859122753, 0.020416846498847008, 0.018448641523718834, 0.028762036934494972, -0.012622755020856857, -0.0005843109101988375, -0.0028096127789467573, -0.016965927556157112, 0.026203369721770287, -0.0488639697432518, 0.014997722581028938, -0.0005043525598011911, -0.0013621619436889887, 0.0153257567435503, -0.021991411224007607, -0.055214714258909225, -0.007872819900512695, 0.010064088739454746, 0.0040151383727788925, 0.015706276521086693, 0.01072015706449747, -0.004408779554069042, -0.008338629268109798, 0.010707035660743713, -0.04117485135793686, -0.006048950366675854, -0.015758762136101723, 0.009106229059398174, 0.041306063532829285, -0.010247787460684776, -0.001403166214004159, -0.6302455067634583, -0.037946995347738266, -0.006094875279814005, 0.00484506506472826, -0.0004809801175724715, -0.005514254793524742, -0.01711026206612587, -0.0022355529945343733, -0.013672464527189732, 0.001976405968889594, 0.007643196266144514, -0.012793333269655704, -0.004471105989068747, -0.005822606850415468, 0.010201863013207912, -0.022306324914097786, -0.005176379345357418, -0.043615423142910004, -0.0016696939710527658, 0.02359221875667572, -0.009276806376874447, 0.006586926523596048, -0.03017914481461048, -0.014367897063493729, -0.0012186469975858927, -0.0020485734567046165, 0.0023798879701644182, 0.009329291991889477, -0.005665150471031666, 0.001413827296346426, -0.02258187346160412, 0.0225949939340353, 0.028919493779540062, 5.93024305999279e-05, 0.03757959604263306, 0.014617202803492546, -0.007675999775528908, 0.033302031457424164, 0.024038344621658325, 0.03143879398703575, -0.026242734864354134, -0.031465038657188416, 0.016572287306189537, 0.013541251420974731, -0.000305481837131083, 0.007400451228022575, 0.014774659648537636, -0.013298505917191505, -0.0022716366220265627, -0.011802669614553452, -0.019314652308821678, -0.012970471754670143, -0.0033131451345980167, -0.001668873941525817, 0.013764314353466034, 0.0038281588349491358, 0.029155677184462547, 0.01221599243581295, 0.008751952089369297, -0.01264243759214878, -5.105031959828921e-05, -0.0094801876693964, -0.01079232431948185, 0.0017976273084059358, -0.006488515995442867, 0.014499111101031303, 0.014276047237217426, -0.00903406087309122, -0.015365120954811573, -0.027712328359484673, 0.0353752076625824, 0.018763555213809013, -0.02716122940182686, -0.015155179426074028, -0.0007097839843481779, 0.023552853614091873, 0.016283616423606873, -0.006468833889812231, 0.020561182871460915, 0.008863483555614948, 0.011625531129539013, 0.0020862973760813475, -0.0030851615592837334, -0.014039862900972366, 0.02633458375930786, 0.020718639716506004, -0.011966686695814133, -0.004805700853466988, -0.01875043287873268, -0.00389048526994884, 0.017031535506248474, 0.015063329599797726, -0.026137763634324074, -0.05999089032411575, 0.017490781843662262, 0.01021498441696167, -0.003959372639656067, 0.01863234117627144, 0.016388587653636932, -0.025245510041713715, 0.005930858198553324, -0.004152912646532059, 0.02296239323914051, 0.0007015831070020795, 0.021466556936502457, 0.001275232876650989, -0.012314403429627419, 0.00554049713537097, 0.026098398491740227, -0.039574041962623596, -0.00969013012945652, -0.014433504082262516, -0.034351740032434464, 0.0016450914554297924, -0.016401708126068115, -0.031097641214728355, 0.020705517381429672, 0.010595504194498062, -0.00567499129101634, -0.02516678161919117, 0.032147347927093506, -0.009381777606904507, 0.013022957369685173, -0.0060259876772761345, 0.005366639234125614, 0.0018566735088825226, -0.004320210311561823, -0.02111227996647358, -0.010687354020774364, 0.04248698800802231, -0.002949027344584465, -0.002343804109841585, 0.02174210548400879, -0.008883165195584297, 0.0026341143529862165, 0.002396289724856615, 0.006062071770429611, 0.006790307350456715, -0.00554049713537097, -0.019288409501314163, -0.01943274401128292, -0.012327524833381176, 0.013593736104667187, 0.008961893618106842, -0.05017610639333725, -0.017451418563723564, -0.0019255606457591057, -0.006521319504827261, 0.006383545231074095, 0.017831938341259956, -0.032462261617183685, 0.004356293939054012, -0.024471350014209747, 0.022030776366591454, 0.004746654536575079, 0.0010915336897596717, 0.006249051075428724, -0.0223325677216053, -0.04033508151769638, -0.03059902787208557, 0.006544281728565693, 0.03705473989248276, -0.03075648471713066, 0.02163713425397873, -0.007400451228022575, 0.009611401706933975, -0.0023552854545414448, 0.04400906711816788, -0.03374815732240677, -0.01833054982125759, 0.020679274573922157, -0.040151383727788925, -0.0005285450606606901, 0.0016139281215146184, -0.017438296228647232, 0.028945736587047577, 0.005766841117292643, -0.01810748688876629, -0.005783242639154196, -0.020889217033982277, -0.014289168640971184, 0.00754478620365262, -0.003506685374304652, -0.016913441941142082, 0.024025224149227142, 0.016808470711112022, 0.04689576476812363, 0.0034574803430587053, -0.009145593270659447, -0.020443089306354523, -0.016979049891233444, 0.014236683025956154, 0.0004506369587033987, 0.022148868069052696, -0.011126919649541378, 0.024838747456669807, -0.0030703998636454344, -0.019655808806419373, 0.007223312743008137, 0.033039603382349014, 0.003982334863394499, 0.013383794575929642, 0.016309859231114388, 0.010890734381973743, 0.009722933173179626, -0.025258632376790047, 0.0116911381483078, -0.030730241909623146, 0.013908648863434792, 0.009066864848136902, 0.021755225956439972, -0.010306834243237972, 0.0027735289186239243, -0.04259195923805237, -0.0011038350639864802, 0.02432701550424099, 0.002519302535802126, 0.04560987278819084, -0.011842033825814724, -0.02222759649157524, -0.005704514216631651, 0.005743878427892923, 0.021781468763947487, -0.006111276801675558, 0.007242994848638773, 0.050149865448474884, 0.0043497332371771336, -0.00902750063687563, 0.0055733006447553635, -0.05301032215356827, -0.005592982750386, 0.011048191227018833, -0.009421141818165779, 0.028709551319479942, 0.0046121603809297085, 0.007177387829869986, -0.0029359059408307076, -0.00852888822555542, 0.03574260324239731, 0.009447384625673294, -0.007846578024327755, 0.016913441941142082, -0.0025701478589326143, -0.01124501135200262, 0.011094115674495697, 0.029890473932027817, 0.032619718462228775, -0.009860707446932793, -0.03450919687747955, -0.00041701344889588654, -0.01574563980102539, -0.0013293585507199168, 0.0010234666988253593, 0.005025483667850494, -0.0006892818491905928, 0.00436941534280777, 0.007879381068050861, -0.00908654648810625, 0.02347412519156933, 0.003160609398037195, 0.0024012101348489523, 0.01859297789633274, 0.01737269014120102, 0.015824368223547935, 0.0034476392902433872, -0.008122126571834087, 0.006462273187935352, -0.014682809822261333, -0.009631083346903324, 0.0011661614989861846, -0.016021190211176872, -0.020771123468875885, 0.015876853838562965, -0.02254250831902027, 0.0067509436048567295, -0.011960126459598541, 0.007190509233623743, 0.006993688642978668, 0.02006257139146328, 0.019235923886299133, -0.016651015728712082, -0.04319554194808006, 0.003044157288968563, 0.012484980747103691, -0.00613423902541399, -0.01775320991873741, -0.024523835629224777, -0.009742614813148975, -0.010497094132006168, 0.001212086295709014, 0.0017713846173137426, 0.014879630878567696, 0.009906631894409657, -0.00987382885068655, -0.01880291849374771, -0.0020534938666969538, 0.04075496643781662, -0.016598530113697052, -0.01910470984876156, -0.010169059969484806, 0.009486748836934566, 0.0005478170933201909, -0.00664597237482667, -0.0018353512277826667, 0.016677258536219597, 0.007177387829869986, -0.012944228947162628, -0.017175870016217232, -0.007577589713037014, -0.027528628706932068, -0.007105220574885607, -0.01031995564699173, -0.0005318254115991294, -0.00043464527698233724, 0.022673722356557846, -0.010497094132006168, 0.004546553827822208, 0.0007188048912212253, 0.017556389793753624, 0.012124143540859222, -0.009801661595702171, -0.017359569668769836, -0.013974255882203579, 0.02642643265426159, 0.07510670274496078, 0.0034804425667971373, -0.027371171861886978, 0.005317434202879667, -0.03579508885741234, -0.0037231878377497196, -0.01728084124624729, -0.028105968609452248, 0.007400451228022575, -0.011323739774525166, -0.016047433018684387, 0.004559675231575966, 0.020141297951340675, -0.01901286095380783, 0.020928580313920975, -0.001817309414036572, 0.003198333317413926, -0.00976885762065649, 0.014000498689711094, -0.020206905901432037, -0.008673223666846752, -0.016086796298623085, 0.01585061103105545, 0.0028998220805078745, 0.019616443663835526, -0.027738571166992188, 0.04511126130819321, 0.024969961494207382, -0.008587935008108616, -0.03175370767712593, -0.005999745335429907, 0.02390713058412075, 0.016611650586128235, -0.0033951536752283573, -0.012760529294610023, 0.03584757447242737, 0.01770072430372238, 0.03390561416745186, -0.01323945913463831, -0.0013490405399352312, 0.0005203442415222526, 0.026518283411860466, 0.005077969282865524, -0.014918994158506393, 0.004572796635329723, -0.010602064430713654, -0.007702242583036423, 0.029103191569447517, -0.002689880318939686, -0.017188990488648415, 0.007997473701834679, -0.006901839282363653, -0.031779952347278595, 0.0022404734045267105, 0.001180102932266891, 0.016021190211176872, -0.0068952785804867744, -0.005055006593465805, -0.019052226096391678, 0.0012588311219587922, -0.019091589376330376, -0.031570009887218475, 0.0038019162602722645, -0.0056684305891394615, 0.01847488433122635, -0.00913903210312128, -0.006583645939826965, -0.023041121661663055, -0.03416803851723671, 0.020049449056386948, -0.008220536634325981, -0.026413312181830406, -0.05154072865843773, 0.0012973751872777939, 0.007131462916731834, -0.0016811751993373036, 0.012576830573379993, -0.02195204794406891, 0.0007434074650518596, 0.007236434146761894, -0.020679274573922157, -0.013895527459681034, -0.011389346793293953, -0.03101891279220581, 0.009742614813148975, -0.026098398491740227, -0.006314658094197512, -0.0012301282258704305, -0.007124902214854956, 0.028210939839482307, 0.032094866037368774, -0.012648997828364372, 0.02538984641432762, -0.001132537960074842, 0.010674232617020607, 0.020154420286417007, 0.0317012220621109, 0.007485739886760712, 0.012734286487102509, -0.026255855336785316, 0.007531664799898863, -0.03820941969752312, 0.0004953315947204828, -0.009558916091918945, -0.011592728085815907, 0.0029473870526999235, 0.008994697593152523, 0.016139281913638115, -0.01684783585369587, -0.016126159578561783, 0.010123134590685368, 0.0023946494329720736, -0.0007270057685673237, -0.016729742288589478, -0.006137519609183073, 0.014197319746017456, 0.01485338807106018, -0.011363103985786438, -0.003477162215858698, -0.027922268956899643, 0.020311877131462097, -0.003641179297119379, 0.013869285583496094, 0.012937667779624462, -0.007964669726788998, 0.03180619329214096, 0.0058816527016460896, -0.0023897290229797363, -0.016755985096096992, 0.008010595105588436, 0.016191767528653145, 0.026872560381889343, -0.01854049228131771, -0.011389346793293953, -0.027817297726869583, -0.007380769122391939, -0.011021948419511318, 0.011330300942063332, 0.005707794800400734, -0.014341654255986214, 0.006235929671674967, 0.007872819900512695, -0.00635730242356658, -0.008391113951802254, -0.0017402212833985686, -0.023198576644062996, -0.023461004719138145, 0.00795154832303524, 0.013554371893405914, 0.020088812336325645, -0.004917232319712639, 0.011212208308279514, -0.011015388183295727, -0.015260149724781513, -0.011933883652091026, -0.027738571166992188, -0.00759727181866765, -0.0009414580999873579, 0.023027999326586723, 0.015653790906071663, 0.056159451603889465, 0.0010702115250751376, 0.017674481496214867, 0.03532272204756737, -0.01276709046214819, -0.0015696435002610087, 0.016913441941142082, 0.015785004943609238, -0.02527175284922123, 0.008935650810599327, 0.014814023859798908, 0.0013219777029007673, 0.01163865253329277, 0.0006847713375464082, 0.008535449393093586, 0.006531160324811935, 0.013501887209713459, -0.004966437350958586, -0.005209182854741812, -0.023867767304182053, -0.004799140151590109, 0.02385464496910572, -0.00852888822555542, -0.011769866570830345, -0.010510215535759926, 0.002435653703287244, 0.04513750225305557, 0.013541251420974731, 0.021295979619026184, 0.024773141369223595, 0.019472109153866768, -0.008935650810599327, 0.010300273075699806, 0.01932777464389801, 0.024248287081718445, -0.009631083346903324, -0.005284630693495274, -0.03991520032286644, -0.005061567295342684, 0.026203369721770287, 0.024943718686699867, 0.021033551543951035, 0.012196310795843601, 0.005996464751660824, 0.00020327868696767837, 0.01932777464389801, 0.001047249068506062, -0.010011603124439716, -0.005484731402248144, -0.028184697031974792, -0.01327226310968399, -0.03164873644709587, -0.0080565195530653, -0.01964268647134304, -0.009939435869455338, 0.008778194896876812, -0.0018566735088825226, 0.022923028096556664, -0.022870542481541634, -0.03781577944755554, 0.014171076938509941, 0.017517024651169777, 0.03259347751736641, 0.016874078661203384, 0.017149627208709717, 0.038944218307733536, -7.806188659742475e-05, -0.02027251198887825, 0.027974754571914673, -0.010050967335700989, 0.006180163938552141, 0.0052419863641262054, 0.023605339229106903, -0.02242441661655903, -0.020403726026415825, 0.022135745733976364, -0.034299254417419434, 0.003337747883051634, -0.04755183309316635, -0.006649252958595753, 0.016401708126068115, 0.00795154832303524, -0.029050707817077637, -0.018501127138733864, -0.008102444000542164, -0.008988136425614357, -0.0015023965388536453, 0.0022191512398421764, 0.0010816927533596754, -0.003959372639656067, -0.0058455690741539, 0.006524599622935057, -0.003091722261160612, 0.005110772326588631, -0.011717380955815315, -0.013895527459681034, 0.013829921372234821, -0.017018413171172142, -0.03721219673752785, -0.004631842486560345, 0.02238505333662033, 0.020626788958907127, -0.008030276745557785, 0.024077709764242172, 0.021886439993977547, 0.005258387885987759, -0.010070648975670338, -0.007741606794297695, -0.018448641523718834, 0.029575562104582787, -0.014171076938509941, -0.010602064430713654, -0.007944988086819649, 0.004205398261547089, 0.0014048063894733787, -0.014184198342263699, -0.008968454785645008, -0.012393130920827389, -0.011881398037075996, 0.0015827649040147662, 0.021991411224007607, 0.0012202871730551124, -0.01287862192839384, 0.0026291939429938793, -0.005051726475358009, 0.00858137384057045, -0.001349860685877502, 0.0049762786366045475, 0.006875596474856138, -0.032199833542108536, -0.0007110140868462622, -0.003985615447163582, -0.005921016912907362, 0.02105979435145855, 0.013042639009654522, -0.014013620093464851, 0.007840016856789589, 0.013934891670942307, -0.002811252838000655, -0.002538984641432762, 0.030625270679593086, 0.000411682907724753, -0.012970471754670143, 0.006170323118567467, -0.002396289724856615, -0.005343677010387182, 0.020298754796385765, -0.012445616535842419, -0.018461763858795166, -0.019918235018849373, -0.00907998625189066, 0.016651015728712082, -0.00542240496724844, -0.0022831179667264223, -0.01827806420624256, 0.007623514160513878, -0.013173853047192097, -0.01363310031592846, -0.013672464527189732, 0.005360078532248735, -0.0076825604774057865, -0.006249051075428724, 0.018606098368763924, -0.012498102150857449, 0.013173853047192097, -0.00863385945558548, 0.0022240716498345137, 0.007787531241774559, -0.023985859006643295, 0.005035324487835169, 0.007774410303682089, 0.004789298865944147, -0.003172090509906411, -0.0033033040817826986, -0.007000249344855547, 0.005514254793524742, -0.029339376837015152, -0.004927073605358601, -0.01579812541604042, 0.006235929671674967, -0.004704010207206011, 0.023762796074151993, 0.02469441294670105, 0.0006454072427004576, -0.00800403393805027, -0.026400191709399223, -0.010254348628222942, -0.00501564284786582, -0.018816040828824043, -0.020731760188937187, -0.011559924110770226, 2.1399104298325256e-05, 0.021807711571455002, -0.01237344928085804, -0.014499111101031303, -0.023093605414032936, -0.010116574354469776, -0.015666913241147995, -0.0018632340943440795, 0.009250563569366932, -0.0048286630772054195, -0.010851371102035046, -0.001753342687152326, 0.028105968609452248, 0.012360327877104282, 0.012091339565813541, 0.00992631446570158, 0.033302031457424164, -0.0016090077115222812, 0.005865251179784536, -0.028184697031974792, -0.01927528902888298, -0.03369567170739174, -0.03611000254750252, 0.010234666056931019, 0.011494318023324013, -0.008732269518077374, 0.018934132531285286, -0.018816040828824043, -0.001812388887628913, -0.0009611401474103332, -0.002730884589254856, 0.011828912422060966, -0.0023798879701644182, 0.019249046221375465, -0.016231130808591843, -0.0014072665944695473, -0.0060292682610452175, 0.013974255882203579, -0.007433254737406969, -0.003444358939304948, 0.014446625486016273, 0.010129695758223534, -0.00402825977653265, -0.028184697031974792, 0.019039103761315346, 0.003513246076181531, -0.006855914369225502, 0.03044157102704048, -0.009519551880657673, -0.0133969159796834, 0.02284429967403412, 0.015076451003551483, 0.014171076938509941, -0.002360205864533782, 0.00863385945558548, -0.015627548098564148, -0.02538984641432762, 0.030625270679593086, -0.016559164971113205, 0.03500780835747719, -0.0029523076955229044, 0.011822352185845375, -0.020233148708939552, 0.005317434202879667, 0.0037658323999494314, 0.004526871722191572, -0.028893250972032547, 0.016441073268651962, -0.000984102487564087, -0.04424525052309036, 0.010064088739454746, -0.0025012607220560312, -0.005409283563494682, -0.01561442669481039, -0.0080565195530653, 0.021466556936502457, 0.0034574803430587053, -0.0025422649923712015, -0.007157705724239349, -0.01859297789633274, -0.010274030268192291, -0.005484731402248144, 0.009355534799396992, -0.024786261841654778, 0.0036543007008731365, 0.23576472699642181, -0.028237182646989822, 0.007407011929899454, 0.005773401353508234, 4.6949891839176416e-05, 0.009598280303180218, 0.009893510490655899, 0.005921016912907362, -0.009545794688165188, -0.018566735088825226, 0.011815791018307209, 0.011310618370771408, -0.04600351303815842, -0.0009611401474103332, 0.0016139281215146184, -0.01498460117727518, -0.008994697593152523, -0.034299254417419434, -0.011618970893323421, -0.02165025658905506, 0.006777185946702957, 0.0014236683491617441, -0.00829270388931036, -0.033459484577178955, 0.030572785064578056, 0.006232649553567171, -0.007557907607406378, 0.02558666653931141, 0.021873319521546364, 0.023106727749109268, -0.03618872910737991, -0.006682056467980146, 0.01045116875320673, 0.0009168555261567235, -0.021072914823889732, -0.011559924110770226, 0.003391873324289918, -0.006442591082304716, 0.012360327877104282, 0.03802572190761566, 0.012681801803410053, -0.015929339453577995, 0.004267724696546793, -0.0024520554579794407, 0.012589951977133751, 0.0022503144573420286, -0.031570009887218475, -0.009972238913178444, -0.0009980440372601151, 0.0002644775668159127, -0.01927528902888298, -0.025455452501773834, 0.02102043107151985, 0.034981563687324524, 0.008935650810599327, -0.007807213347405195, -0.00776784960180521, 0.04238201677799225, 0.02058742567896843, -0.00020748161477968097, -0.00040655737393535674, 0.03813069313764572, -0.006954324431717396, 0.010575821623206139, 0.004221799783408642, -0.005048445891588926, -0.009178396314382553, 0.0031179648358374834, -0.00309336232021451, -0.01537824235856533, 0.013232898898422718, 0.0015155179426074028, -0.031097641214728355, -0.014459746889770031, -0.039731498807668686, -0.013843042775988579, 0.04285438358783722, 0.03521775081753731, 0.029181919991970062, 0.030940184369683266, -0.0007278258563019335, -0.007840016856789589, 0.002227352000772953, -0.033616941422224045, 0.008574813604354858, -0.0296018049120903, 0.03621497377753258, -0.020390605553984642, -0.0005761100328527391, -0.012891743332147598, -0.003408275078982115, -0.032882146537303925, -0.002917864127084613, 0.003591974265873432, -0.011999490670859814, 0.015260149724781513, 0.02716122940182686, 0.005858690477907658, 0.013154170475900173, -0.015889976173639297, -0.010254348628222942, 0.012045415118336678, 0.03495532274246216, 0.0076825604774057865, -0.018881646916270256, 0.007216752041131258, 0.016204888001084328, 0.009250563569366932, 0.0009119349997490644, -0.03185867890715599, 0.002509461482986808, -0.013869285583496094, 0.01197980809956789, 0.007538225501775742, -0.00024500052677467465, 0.007308601401746273, -0.006180163938552141, 0.004566235933452845, 0.011914201080799103, -0.0049762786366045475, -0.00759727181866765, -0.00937521643936634, -0.006665654480457306, -0.008758512325584888, -0.0073479656130075455, -0.03579508885741234, -0.02601967193186283, 0.008358310908079147, -0.02301487885415554, -0.0006650892901234329, 0.025022447109222412, -0.005100931506603956, 0.027056260034441948, -0.001674614497460425, -0.01779257319867611, -0.02238505333662033, 0.0020272512920200825, -0.001238328986801207, -0.011487756855785847, 0.005104211624711752, -0.03981022909283638, 0.01003128569573164, -0.0019058785401284695, -0.011723941192030907, 0.00632121879607439, -0.02222759649157524, 0.0032426179386675358, 0.005747159011662006, 0.014092348515987396, -0.015456970781087875, -0.0032491786405444145, 0.023146091029047966, -0.023684067651629448, -0.011559924110770226, 0.015575062483549118, 0.0107726426795125, -0.02252938784658909, -0.0317012220621109, -0.0008496085065416992, -0.004418620374053717, -0.020705517381429672, 0.017031535506248474, 0.008017155341804028, -0.004287406802177429, -0.02664949744939804, -0.01316073164343834, -0.16606402397155762, -0.020731760188937187, 0.03839312121272087, -0.03771080821752548, 0.022621236741542816, -0.0032032537274062634, 0.02159777097404003, 0.011901079677045345, -0.03752711042761803, -0.0010693913791328669, -0.0031934126745909452, 0.009303049184381962, -0.010005042888224125, -0.004021699074655771, 0.0017418614588677883, -0.006563963834196329, -0.002263435861095786, 0.0039200084283947945, 0.013829921372234821, 0.003752710996195674, 0.031832437962293625, -0.028105968609452248, 0.011802669614553452, 0.008273022249341011, -0.02453695610165596, 0.01208477932959795, -0.00242253253236413, 0.05106836184859276, 0.018199335783720016, -0.01548321358859539, -0.0025635871570557356, -0.0259671863168478, 0.014381018467247486, -0.02668886072933674, 0.01321977749466896, 0.016073673963546753, 0.0012883542804047465, 0.000818445289041847, -0.025560423731803894, 0.023894010111689568, 0.013987377285957336, 0.007918745279312134, 0.006472114473581314, 0.003952811937779188, -0.00443502189591527, 0.010096891783177853, 0.02221447415649891, -0.009834464639425278, 0.00658036582171917, -0.02121725119650364, 0.021414071321487427, -0.019983842968940735, 0.007118341512978077, -0.0005215743440203369, 0.027686085551977158, 0.005317434202879667, -0.012156946584582329, 0.02027251198887825, 0.033092088997364044, -0.011947005055844784, -0.02432701550424099, 0.0022470341064035892, -0.0018632340943440795, -0.017595753073692322, -0.011428711004555225, -0.010759521275758743, -0.023211698979139328, 0.0011874837800860405, -0.029890473932027817, -8.774914022069424e-05, -0.006337620317935944, 0.00443502189591527, -0.017293961718678474, -0.005530656315386295, -0.0004416160227265209, -0.0013572414172813296, -0.04253947362303734, 0.010379001498222351, 0.029103191569447517, -0.008049958385527134, -0.011756745167076588, 0.025573544204235077, -0.004595758859068155, 0.006777185946702957, 0.01852736994624138, -0.012452177703380585, -0.007380769122391939, 0.00036883342545479536, -0.03542769327759743, -0.004894270095974207, 0.003719907719641924, -0.03692352771759033, -0.011664895340800285, -0.010805445723235607, 0.011146601289510727, -0.010431487113237381, 0.01842239871621132, -0.00771536398679018, 0.02464192733168602, -0.028263425454497337, 0.004999240860342979, 0.024235164746642113, -0.011999490670859814, 0.01264243759214878, 0.003209814429283142, 0.012740847654640675, -0.01627049595117569, 0.012484980747103691, 0.03881300240755081, 0.007092099171131849, -0.020259391516447067, 0.004179155454039574, 0.018396155908703804, 0.018934132531285286, 0.000929976929910481, 0.022148868069052696, -0.007315162103623152, -0.015417606569826603, 0.010155938565731049, 0.012629316188395023, 0.057524073868989944, 0.017818816006183624, -0.006583645939826965, 0.0011555004166439176, -0.0034410785883665085, -0.04860154539346695, -0.07500173151493073, -0.02290990762412548, 0.004182435572147369, 0.026308340951800346, 0.003657581051811576, 0.006252331659197807, -0.01503708679229021, 0.023657824844121933, -0.032199833542108536, 0.019787020981311798, -0.01733332686126232, -0.025573544204235077, -0.0005674991407431662, -0.002281477674841881, 0.015758762136101723, -0.016506679356098175, 0.00496315723285079, 0.017359569668769836, -0.021361585706472397, 0.03443046659231186, -0.0070855384692549706, -0.0023684068582952023, 0.0021879880223423243, -0.00030220148619264364, -0.00937521643936634, 0.00034361580037511885, -0.04070248082280159, 0.02284429967403412, 0.010634868405759335, -0.014932115562260151, 0.010182181373238564, -0.037684567272663116, -0.013659343123435974, -0.03595254570245743, 0.003667422104626894, -0.012235675007104874, -0.0037231878377497196, -0.013449401594698429, 0.016821593046188354, -0.040203869342803955, -0.011933883652091026, -0.023789038881659508, 0.012668680399656296, -0.014879630878567696, -0.031045155599713326, 0.006711579393595457, -0.00931617058813572, 0.029155677184462547, 0.00969013012945652, -0.03091394156217575, -0.008187733590602875, -0.021256614476442337, -0.026610132306814194, 0.028184697031974792, 0.02996920235455036, 0.02254250831902027, -0.01815997250378132, -0.008837240748107433, -0.02105979435145855, -0.016257373616099358, 0.009972238913178444, -0.010050967335700989, -0.03471913933753967, 0.025717880576848984, 0.02954931929707527, -0.014774659648537636, -0.00885036215186119, -0.004825382959097624, 0.007498861290514469, -0.021715862676501274, -0.004746654536575079, 0.03101891279220581, -0.007144584320485592, 0.017149627208709717, -0.021046673879027367, -0.0006823110743425786, -0.019865749403834343, -0.002509461482986808, 0.019367137923836708, -0.008509206585586071, 0.0004399758472573012, -0.028079725801944733, 6.565808871528134e-05, 0.0161130391061306, 0.008233658038079739, -0.004940194543451071, -0.025927821174263954, -0.006659093778580427, 0.0065344409085810184, -0.032619718462228775, 0.007623514160513878, 0.008423917926847935, 0.0039003263227641582, -0.005678271874785423, -0.012583390809595585, 0.0020698956213891506, -0.0002220381429651752, -0.015863733366131783, 0.009880389086902142, 0.0519343726336956, -0.014512232504785061, -0.004057782702147961, -0.07573653012514114, 0.025822849944233894, -0.005038605071604252, -0.020469332113862038, -0.0036018153186887503, -0.01959020085632801, -0.0003032265813089907, 0.002002648776397109, 0.005579861346632242, 0.011054751463234425, -0.021768348291516304, 0.008443599566817284, -0.016821593046188354, -0.0003680133377201855, 0.004175875335931778, -0.012301282025873661, 0.010149377398192883, -0.018501127138733864, 0.008135247975587845, -0.0027718888595700264, -0.009198077954351902, 0.016139281913638115, 0.014420382678508759, 0.018868526443839073, -0.017976272851228714, -0.0031114041339606047, -0.01827806420624256, 0.023316670209169388, -0.00861417781561613, -0.005996464751660824, 0.019682051613926888, -0.017595753073692322, -0.0008012235048227012, 0.048680271953344345, 0.00462200166657567, -0.018999740481376648, -0.00987382885068655, 0.011632092297077179, 0.023290427401661873, 0.04309057071805, -0.02401210181415081, -0.041096121072769165, 0.01747766137123108, -0.011553363874554634, -0.020285634323954582, 0.010621747002005577, 0.011540242470800877, -0.013029517605900764, 0.01306232064962387, 0.015168300829827785, 0.011815791018307209, 0.018829161301255226, -0.008351750671863556, -0.0153257567435503, -0.00334922899492085, 0.0032491786405444145, 0.05007113888859749, -0.0003089671954512596, 0.02991671673953533, -0.00861417781561613, 0.04080745205283165, 0.03212110698223114, 0.0012612914433702826, 0.0005777502083219588, -0.0053075929172337055, 0.017188990488648415, -0.012248796410858631, 0.002598030725494027, 0.015981825068593025, -0.03185867890715599, 0.013698707334697247, 0.001116136321797967, 0.011363103985786438, 0.019406501203775406, 0.031884923577308655, -0.0023339632898569107, -0.009591719135642052, 0.006941203027963638, -0.02954931929707527, 0.0179500300437212, 0.017412053421139717, -0.009440823458135128, 0.008220536634325981, 0.01805500127375126, 0.008286143653094769, 0.030835213139653206, -0.007761288899928331, 0.005606104154139757, -0.0001661698188399896, 0.009755736216902733, 0.010654550045728683, 0.00525182718411088, -0.01875043287873268, 0.006065351888537407, 0.0054552084766328335, 0.031360067427158356, -0.007092099171131849, -0.0015721038216724992, 0.008784755133092403, 0.017503904178738594, 0.028814522549510002, -0.012111022137105465, 0.00542240496724844, -0.016572287306189537, -0.026045914739370346, 0.019078467041254044, -0.017674481496214867, -0.023251062259078026, 0.005005801562219858, 0.015680033713579178, 0.013121367432177067, -0.02563915215432644, -0.014590959995985031, -0.006183444522321224, -0.029942959547042847, 0.013895527459681034, 0.010405244305729866, -0.002822734182700515, -0.033511970192193985, 0.029811745509505272, 0.01543072797358036, -0.005396162159740925, 0.01579812541604042, -0.007721924688667059, 0.015286392532289028, 0.048680271953344345, 0.028315911069512367, -0.0053108735010027885, 0.023710310459136963, 0.0011120358249172568, -0.0046121603809297085, 0.004152912646532059, 0.008712587878108025, -0.02095482312142849, -0.04311681166291237, -0.012452177703380585, -0.009257124736905098, 0.029628047719597816, 0.011933883652091026, 0.09536609798669815, 0.005005801562219858, 0.004231641069054604, 0.01637546718120575, -0.032462261617183685, 0.0031917726155370474, 0.016559164971113205, 0.03545393422245979, -0.008187733590602875, 0.00010425336222397164, -0.0024274529423564672, 0.0023733272682875395, -0.02427452988922596, -0.006032548379153013, -0.01726771891117096, 0.013685585930943489, -0.02296239323914051, 0.020351240411400795, -0.014236683025956154, -0.015666913241147995, 0.01932777464389801, -0.0023520051036030054, 0.01480090245604515, 0.001588505576364696, -0.012019172310829163, -0.02016754075884819, 0.028604580089449883, 0.01363310031592846, -0.006334340199828148, -0.02674134634435177, -0.00903406087309122, 0.008758512325584888, -0.047368135303258896, -0.01890788972377777, 0.006340900901705027, -0.012517784722149372, 0.009919753298163414, -0.0017943469574674964, 0.010260908864438534, -0.0038609623443335295, 0.005625786259770393, 0.014971479773521423, 0.0019157195929437876, -0.00958515889942646, -0.01679535023868084, 0.007407011929899454, -0.00048385042464360595, -0.01770072430372238, -0.02163713425397873], '_distance': 0.3938315510749817}),\n",
+ " Document(page_content='The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.\\n2020 Update: I’ve created a “Narrated Transformer” video which is a gentler approach to the topic:', metadata={'vector': [-0.02712874673306942, -0.005967519711703062, 0.029046399518847466, -0.01589103601872921, 0.029260961338877678, 0.0017349390545859933, -0.007067152764648199, -0.015327810309827328, -0.01750025525689125, -0.0014390774304047227, 0.027571281418204308, 0.020370028913021088, -0.014415918849408627, 0.0084215784445405, 0.010935982689261436, -0.012665892951190472, 0.013054787181317806, 0.006879410240799189, -0.011834463104605675, -0.016159238293766975, -0.019069243222475052, 0.015233938582241535, -0.006835827603936195, -0.012397689744830132, -0.017218641936779022, 0.013007852248847485, 0.03218437731266022, -0.03462502360343933, -0.011552849784493446, -0.018640117719769478, 0.0340886190533638, 0.003489993279799819, -0.01646767184138298, -0.02754446119070053, -0.01162660587579012, -0.01054038293659687, -0.00021278146596159786, -0.011914924718439579, 0.029073219746351242, -0.02346777357161045, 0.022743625566363335, 0.013249235227704048, -0.00035390243283472955, -0.01778186857700348, -0.02378961816430092, 0.0046767923049628735, -0.007536508142948151, -0.03462502360343933, -0.0038855932652950287, 0.020437078550457954, 0.02454058639705181, 0.02565363049507141, -0.021523302420973778, -0.025506118312478065, -0.02100030519068241, 0.0025646924041211605, 0.0003545310173649341, 0.03416907787322998, 0.011914924718439579, -0.005645676050335169, 0.007020216900855303, -0.006410054862499237, -0.016293341293931007, 0.010942688211798668, -0.020571179687976837, -0.012458035722374916, -0.0013276055688038468, 0.029824187979102135, 0.0008469352032989264, 0.00779130170121789, 0.03046787716448307, 0.02800040692090988, 0.004096802789717913, -0.014402508735656738, 0.03784346207976341, -0.018640117719769478, -0.009340174496173859, -0.011244416236877441, -0.008944574743509293, 0.006094916258007288, 0.005541747435927391, -0.0025797788985073566, 0.015421681106090546, 0.03076290152966976, 0.007932107895612717, 0.005592035595327616, 0.022609524428844452, 0.006792244501411915, -0.010580613277852535, 0.01090916246175766, 0.016655415296554565, 0.0058032451197505, 0.0050086937844753265, 0.01682974584400654, 0.025988884270191193, 0.04299296438694, -0.011103609576821327, 0.032130736857652664, 0.003949291072785854, -0.0274640005081892, -0.004264429677277803, 4.4709253415931016e-05, -0.023829849436879158, -0.006614559795707464, -0.005535042379051447, -0.018465785309672356, 0.006912935990840197, 0.004958405625075102, -0.01732592284679413, 0.023601876571774483, 0.006684963125735521, 0.028831837698817253, 0.0010267151519656181, -0.036985211074352264, 0.007516392972320318, -0.0010267151519656181, -0.009662018157541752, -0.009333468973636627, -0.004680144600570202, -0.005605445709079504, 0.051521822810173035, 0.011472389101982117, -0.00039245662628673017, -0.02016887627542019, 0.010426396504044533, 0.023038649931550026, 0.0017869033617898822, -0.009930221363902092, -0.005400940775871277, -0.03116520680487156, -0.015233938582241535, 0.02016887627542019, 0.010352641344070435, 0.0022411723621189594, -0.014295226894319057, 0.02971690706908703, -0.011110315099358559, -0.015502141788601875, -0.01947154849767685, -0.010594023391604424, 0.006396644748747349, -0.010526972822844982, -0.006785539444535971, -0.013604603707790375, -0.0035972746554762125, 0.03387405723333359, -0.00011733887367881835, 0.03623424470424652, 0.024875840172171593, 0.023333672434091568, 0.011579670011997223, -0.006309478543698788, 0.0015086426865309477, -0.005303717218339443, 0.023722566664218903, 0.03730705752968788, 0.003831952577456832, 0.008575795218348503, -0.01420135609805584, 0.013410156592726707, 0.014268406666815281, 0.004421999212354422, 0.0004982711398042738, -0.022126758471131325, 0.019914083182811737, 0.021643992513418198, 0.00877024233341217, -0.010151488706469536, 0.0005615503177978098, -0.006172024644911289, -0.005669143982231617, 0.02140261046588421, -0.0287781972438097, 0.013302875682711601, 0.0028245141729712486, -0.012283703312277794, 0.018787629902362823, 0.0005766367539763451, -0.017299102619290352, -0.0072683049365878105, 0.02523791417479515, -0.011023148894309998, 0.014925504103302956, 0.014013613574206829, -0.02354823611676693, -0.01650790311396122, 0.000825562747195363, -0.017138181254267693, 0.00817349087446928, -0.012109371833503246, 0.011237711645662785, 0.04154466465115547, 0.007945518009364605, -0.010533678345382214, -0.6230895519256592, -0.02100030519068241, 0.011948449537158012, -0.023829849436879158, -0.003188264789059758, -0.017687996849417686, -0.013799051754176617, 0.01741979271173477, -0.015609423629939556, 0.02551952749490738, -0.004492402542382479, -0.017956199124455452, 0.0014189622597768903, -0.0021087471395730972, -0.0015647977124899626, -0.016601774841547012, -0.0057261367328464985, -0.03510779142379761, 0.007858351804316044, 0.01984703168272972, -0.03280124440789223, 0.030494699254631996, -0.044199876487255096, -0.012478150427341461, 0.008515449240803719, -0.008843998424708843, -0.001840544049628079, -0.0030239904299378395, 0.0026334195863455534, 0.015341220423579216, -0.03218437731266022, 0.006805654615163803, 0.029690086841583252, -0.010178308933973312, 0.02104053646326065, -0.007798006758093834, 0.0024523823522031307, 0.04039139300584793, 0.002227762248367071, 0.02028956636786461, -0.02334708347916603, -0.022166989743709564, 0.019900672137737274, 0.015260759741067886, 0.008937869220972061, 0.005866943392902613, 0.019042422994971275, 0.015756934881210327, -0.013108428567647934, -0.018465785309672356, -0.022462012246251106, 0.00790528766810894, -0.0001966473791981116, -0.014523199759423733, 0.00019277099636383355, 0.0011792556615546346, 0.03660972788929939, -0.0010510210413485765, 0.009051855653524399, -0.02597547322511673, -0.010144783183932304, -0.00815337523818016, 0.003630799939855933, 0.0012337344232946634, -0.019619058817625046, -0.0009638550691306591, 0.012940801680088043, 0.03025331348180771, -0.0076102642342448235, -0.02770538441836834, 0.01302796695381403, 0.041437383741140366, -0.00913902185857296, -0.015917856246232986, -0.0003306441649328917, 0.018720578402280807, 0.006557566579431295, 0.005628913175314665, 0.012833519838750362, 0.027651743963360786, 0.027571281418204308, 0.007147613447159529, -0.01278658490628004, -0.017969610169529915, 0.02112099714577198, 0.008207015693187714, -0.014751172624528408, -0.004499107599258423, -0.0214428398758173, -0.019900672137737274, 0.00633965153247118, 0.023735977709293365, -0.027517640963196754, -0.05680542439222336, -0.007067152764648199, 0.007623674813657999, 0.0014776316238567233, 0.006721841171383858, 0.03033377416431904, -0.008548974990844727, 0.01688338816165924, -0.010339231230318546, 0.027759024873375893, 0.001917652552947402, 0.029046399518847466, 0.0015580925391986966, -0.0035168135073035955, 0.028107687830924988, 0.014992555603384972, -0.01750025525689125, -0.019619058817625046, -0.025747500360012054, -0.011204185895621777, 0.009662018157541752, -0.0030306954868137836, -0.035402812063694, 0.01233063917607069, 0.015703294426202774, -0.007925402373075485, -0.021214868873357773, 0.015475321561098099, 0.015073016285896301, 0.009822939522564411, 0.012712828814983368, 0.016078777611255646, 0.01072141993790865, 0.005535042379051447, -0.021992657333612442, -0.02136237919330597, 0.015877624973654747, -0.003367625642567873, -0.00798574835062027, 0.024473536759614944, 0.010493448004126549, 0.006765424273908138, 0.02947552502155304, 0.0103928716853261, -0.015837395563721657, 0.0011549497721716762, -0.021831735968589783, -0.01198197528719902, -0.0027256144676357508, 0.021228278055787086, -0.005367415025830269, -0.0358051173388958, -0.020557770505547523, -0.013758820481598377, 0.00012980612518731505, 0.004639914259314537, -0.0022646402940154076, -0.00384536269120872, 0.0036039797123521566, -0.01700407825410366, -0.002854687161743641, -0.010185014456510544, -0.01823781244456768, 0.007254894822835922, -0.03819212689995766, -0.028456352651119232, -0.027893126010894775, 0.008267361670732498, 0.04186651110649109, -0.02149648219347, 0.027812665328383446, -0.02663257159292698, -0.012169716879725456, -0.010962802916765213, 0.03915765881538391, -0.013678359799087048, -0.049966245889663696, 0.029797367751598358, -0.03902355581521988, -0.0070872679352760315, -0.01564965210855007, 0.006222312804311514, 0.010205129161477089, -0.003986169118434191, -0.003915765788406134, -0.00266526872292161, -0.022783856838941574, -0.023025238886475563, 0.010325821116566658, -0.03207709640264511, -0.008495334535837173, 0.023575056344270706, 0.009534621611237526, 0.0203834380954504, -0.017433203756809235, -0.02840271219611168, 0.012639072723686695, -0.000759350135922432, 0.0075298030860722065, -0.0043683587573468685, 0.01762094534933567, -0.0363147035241127, 0.010520268231630325, 0.020155465230345726, 0.0009948660153895617, 0.005072391591966152, 0.03588557988405228, 0.015354630537331104, 0.01340345200151205, 0.0049047647044062614, -0.029368244111537933, 0.01585080474615097, 0.010453217662870884, 0.009863169863820076, -0.03816530480980873, -0.0019461491610854864, 0.0043515958823263645, 0.018707169219851494, -0.008931164629757404, -0.0188144501298666, -0.025801140815019608, -0.020557770505547523, 0.041276462376117706, 0.0027474057860672474, 0.0327744223177433, -0.024393076077103615, -0.012994442135095596, 0.010889047756791115, -0.010647664777934551, -0.006296068429946899, -0.0021456251852214336, -0.011485799215734005, 0.01531440019607544, 0.011908219195902348, -0.004066630266606808, 0.008931164629757404, -0.016561543568968773, -0.015140066854655743, 0.015502141788601875, 0.008736717514693737, 0.021831735968589783, 0.008575795218348503, -0.021751273423433304, 0.01225017849355936, -0.002735672052949667, 0.022220630198717117, 0.001709794974885881, 0.0001999999221879989, -0.00014971183554735035, -0.0044152941554784775, -0.022998418658971786, -0.0002587741182651371, 0.01939108595252037, 0.04122282192111015, -0.007301830220967531, -0.025640219449996948, -0.013007852248847485, -0.007295125164091587, 0.004485697485506535, 0.013430272229015827, 0.00607144832611084, 0.006403349805623293, 0.0046600294299423695, 0.004572863690555096, 0.008327707648277283, 0.017674585804343224, 0.030065571889281273, -0.003932528663426638, 0.019605649635195732, 0.01092927809804678, 0.0011859607184305787, 0.0001876374299172312, -0.005558509845286608, 0.010748241096735, -0.014147715643048286, 0.00192268134560436, -0.0004362491599749774, 0.0018522778991609812, -0.014845043420791626, 0.006547509226948023, -0.0318625308573246, 0.022327911108732224, 0.025827961042523384, -0.000696909090038389, 0.03510779142379761, 0.008300887420773506, 0.020316386595368385, -0.0009730745805427432, -0.0350005105137825, 0.009098791517317295, 0.004059924744069576, -0.006835827603936195, -0.027732204645872116, -0.007053742650896311, -0.01408066414296627, -0.0296364463865757, 0.012726238928735256, 0.0009110525134019554, -0.00456951092928648, 0.000693975598551333, 0.00817349087446928, -0.020691871643066406, -0.005833418108522892, 0.009346879087388515, -0.01481822319328785, 0.004847771488130093, -0.020410258322954178, 0.01000397652387619, 0.001646096701733768, -0.02197924628853798, -0.017272282391786575, 0.017875738441944122, -0.0017584067536517978, -0.005099212285131216, -0.008797062560915947, -0.025921832770109177, -0.02424556389451027, 0.021523302420973778, 0.016481082886457443, 0.010205129161477089, -0.03255986049771309, 0.013269349932670593, -0.019860442727804184, -0.0008054475183598697, 0.010325821116566658, 0.01843896508216858, -0.008696486242115498, -0.0024875840172171593, 0.002286431845277548, 0.015944676473736763, 0.03180889040231705, 0.07359494268894196, 0.023534825071692467, -0.012002089992165565, 0.02173786424100399, -0.029877828434109688, 0.005756309721618891, -0.03178207203745842, -0.04323434457182884, 0.015984907746315002, 0.004834361374378204, -0.017071129754185677, -0.010117962956428528, 0.021791504696011543, -0.02643141895532608, 0.026297317817807198, 0.016293341293931007, -0.016816336661577225, -0.01367165520787239, 0.013839282095432281, -0.01955200918018818, 0.013530848547816277, 0.0010518591152504086, 0.020142056047916412, -0.002470821375027299, 0.019873851910233498, -0.0170443095266819, 0.017299102619290352, 0.027396950870752335, 0.0032905172556638718, -0.022985009476542473, 0.01500596571713686, -0.0011029853485524654, -0.02181832492351532, 0.0014558401890099049, -0.013329695910215378, 0.032211195677518845, 0.004851124249398708, 0.032962165772914886, 0.000758092908654362, 0.011049969121813774, -0.006772129330784082, 0.018291454762220383, 0.014603660441935062, -0.0073487660847604275, -0.02378961816430092, -0.0135040283203125, -0.004197379108518362, 0.019578829407691956, 0.0007865895167924464, -0.023212980479002, -0.008133260533213615, 0.00395264383405447, -0.05562533065676689, 0.028375891968607903, 0.0025747499894350767, -0.011606490239501, -0.014040433801710606, -0.002336719771847129, -0.0067084310576319695, 0.0073487660847604275, -0.0021892080549150705, -0.01621287874877453, 0.009662018157541752, -0.0002994236710947007, 0.040659595280885696, -0.0052199033088982105, -0.005964167416095734, 0.002539548324421048, -0.011901513673365116, 0.00823383592069149, -0.003630799939855933, -0.00648381095379591, -0.05895105004310608, 0.01412089541554451, 0.02454058639705181, 0.016253110021352768, 0.027598103508353233, -0.009648608043789864, -0.012806699611246586, 0.004861181601881981, 0.00341623742133379, -0.028644094243645668, 0.001261392841115594, -0.027061697095632553, 0.002663592342287302, -0.0038218949921429157, -0.030199673026800156, -0.01500596571713686, -0.0006059714360162616, 0.016776105388998985, 0.0022311147768050432, 0.006054685916751623, 0.010875636711716652, -0.013081608340144157, 0.008093029260635376, 0.03601968288421631, 0.011163955554366112, 0.029207320883870125, -0.0005380825605243444, -0.030360594391822815, 0.00384536269120872, -0.025908423587679863, -0.005307069513946772, -0.020329797640442848, 0.020557770505547523, 0.014952325262129307, 0.02623026631772518, 0.020799152553081512, -0.016816336661577225, -0.022301090881228447, 0.020477309823036194, -0.02000795304775238, -0.014603660441935062, 0.00126306910533458, 0.015622833743691444, 0.007684020325541496, 0.012739649042487144, 0.014161125756800175, -0.01736615225672722, -0.011311466805636883, -0.004512517713010311, -0.01321570947766304, 0.030494699254631996, 0.019417906180024147, -0.014845043420791626, 0.015233938582241535, 0.0016108950367197394, 0.0028446295764297247, -0.028295431286096573, 0.020142056047916412, 0.009279828518629074, 0.021912196651101112, 0.0005426922580227256, -0.0026367721147835255, -0.027598103508353233, 0.003966053947806358, -0.016695644706487656, -0.009702248498797417, -0.01676269620656967, -0.01843896508216858, 0.009132316336035728, -0.02635095827281475, -0.021456250920891762, -0.01605195738375187, 0.010573908686637878, -0.01798301935195923, -0.022622933611273766, 0.023896899074316025, 0.009105496108531952, 0.02939506433904171, -0.019873851910233498, 0.01107678934931755, -0.028831837698817253, -0.017835509032011032, 0.019176524132490158, -0.008428283967077732, 0.018210992217063904, 0.012384279631078243, 0.020034775137901306, 0.030816541984677315, 0.04776697978377342, 0.0013753791572526097, -0.0001487689296482131, 0.035912398248910904, -0.003999579232186079, 0.004757252987474203, 0.0021204811055213213, 0.020812563598155975, -0.021188046783208847, 0.011928334832191467, 0.02305205911397934, 0.006192139815539122, 0.018546245992183685, 0.013343106023967266, 0.023896899074316025, 0.00017307483358308673, -0.0050958595238626, -0.0075767384842038155, -0.008924459107220173, -0.027866305783391, -0.00546463904902339, 0.03038741648197174, -0.015435091219842434, 0.0048578293062746525, -0.02885865792632103, -0.006795596797019243, 0.027759024873375893, 0.01737956330180168, 0.02985100820660591, 0.007462752051651478, 0.024258973076939583, 0.010708009824156761, -0.013108428567647934, 0.018264632672071457, 0.028375891968607903, -0.018492605537176132, -0.003939233720302582, -0.02993147075176239, 0.0033961222507059574, 0.030146032571792603, 0.0024959654547274113, 0.003557044081389904, -0.004331480711698532, 0.0041370331309735775, -0.0004890516283921897, 0.014067254029214382, -0.000573703262489289, -0.025506118312478065, -0.0024054469540715218, -0.010634254664182663, -0.026337547227740288, -0.041973792016506195, -0.021711044013500214, 0.0034531154669821262, -0.006979986559599638, 0.0022428487427532673, 0.000508747820276767, 0.0023886840790510178, -0.02458081766963005, -0.042134713381528854, 0.005079096648842096, 0.0086227310821414, 0.03304262459278107, 0.01420135609805584, 0.0205845907330513, 0.015716703608632088, -0.00329889846034348, -0.026900773867964745, 0.009963746182620525, -0.002157358918339014, 0.007080562878400087, 0.0018271338194608688, -0.008186900988221169, -0.022488832473754883, -0.020115235820412636, 0.011492504738271236, 0.01048003789037466, -0.008837292902171612, -0.03910401836037636, 0.015636242926120758, 0.004934937693178654, 0.014536609873175621, -0.02259611338376999, -0.024151692166924477, 0.011593080125749111, 0.017352743074297905, 0.001461707055568695, 0.01794278994202614, 0.006034570746123791, -0.0006407540640793741, -0.02091984450817108, 0.01438909862190485, -0.00043583009392023087, 0.004100155550986528, 0.013490617275238037, -0.014415918849408627, 0.004482344724237919, -0.011666836217045784, -0.0037179659120738506, -0.0103928716853261, 0.021724453195929527, 0.001082870177924633, -0.014027023687958717, 0.04433397948741913, 0.03070926107466221, -0.0007731793448328972, -0.0017533779609948397, 0.01909606344997883, -0.007831531576812267, 0.028027227148413658, -0.03301580622792244, -0.015435091219842434, -0.000507490593008697, -0.021630583330988884, 0.022904546931385994, -0.0322648361325264, 0.0041068606078624725, -0.028724554926156998, -0.015596013516187668, -0.022529063746333122, 0.018345095217227936, 0.010969508439302444, -0.017848918214440346, -0.0007325297920033336, 0.013276055455207825, 0.008119849488139153, -0.007161023560911417, 0.004592978861182928, 0.009869875386357307, -0.016360390931367874, -0.01877421885728836, 0.0021154521964490414, -0.010138078592717648, 0.015663063153624535, -0.006959871388971806, -0.006182082463055849, 0.001090413425117731, 0.03516143187880516, 0.008877524174749851, -0.020718691870570183, 0.035054150968790054, 0.008877524174749851, -0.018291454762220383, 0.009192662313580513, 0.0066916681826114655, -0.022032886743545532, 0.023561645299196243, -0.004257724620401859, -0.008247246034443378, 0.0003327395243104547, 0.011559555307030678, 0.013879512436687946, 0.0043515958823263645, -0.0029669972136616707, -0.030119212344288826, 0.0067788343876600266, 0.0009144050418399274, -0.006487163249403238, 0.001266421633772552, -0.0046600294299423695, 0.00994363147765398, 0.0044622295536100864, 0.016186058521270752, -0.0314602293074131, 0.012297113426029682, -0.0035268713254481554, -0.00032645350438542664, -0.019565418362617493, -0.03846032917499542, 0.011827758513391018, -0.020075004547834396, 0.002601570449769497, -0.006286011077463627, -0.008542269468307495, 0.03416907787322998, 0.014630480669438839, -0.009534621611237526, 0.023440953344106674, 0.0030139328446239233, 0.005424408242106438, 2.97537862934405e-05, 0.021067356690764427, 0.025063583627343178, 0.004482344724237919, -0.02384325861930847, -0.019605649635195732, -0.006346356589347124, -0.0017449966398999095, -0.026498470455408096, -0.036985211074352264, -0.011700361967086792, 0.024232152849435806, 0.007898582145571709, -0.02947552502155304, -0.00710067804902792, -0.03046787716448307, -0.034061796963214874, 0.00139465625397861, -0.004888002295047045, -0.006678258068859577, -0.012323933653533459, 0.01782209798693657, 0.0035939221270382404, 0.016735875979065895, -0.011888103559613228, 0.019914083182811737, 0.0025948653928935528, 0.01885467953979969, -0.011512619443237782, 0.002790988888591528, -0.02086620405316353, -0.00967542827129364, -0.04218835383653641, -0.0038621253333985806, 0.020182285457849503, -0.00041655299719423056, -0.00994363147765398, 0.031594328582286835, 0.0013753791572526097, -0.011827758513391018, -0.014871863648295403, 0.023561645299196243, 0.023816438391804695, 0.014523199759423733, 0.017232051119208336, -0.019686110317707062, 0.0028781548608094454, -0.0030340480152517557, 0.0015547400107607245, -0.01029229536652565, -0.022542472928762436, 0.017956199124455452, 0.0021037182305008173, -0.0023132520727813244, -0.029207320883870125, 0.004073335323482752, 0.008160080760717392, -0.016789516434073448, 0.028536813333630562, 0.006104974076151848, -0.018331684172153473, 0.03162115067243576, 0.010030797682702541, 0.009179252199828625, -0.011485799215734005, 0.012639072723686695, -0.011438863351941109, -0.022086529061198235, 0.03942586109042168, -0.02205970697104931, 0.0084215784445405, -0.00030843360582366586, 0.007456046994775534, -0.0314602293074131, -0.005548452492803335, 0.014322047121822834, 0.005484754219651222, -0.021804915741086006, 0.026538699865341187, -0.015917856246232986, -0.04808882251381874, 0.007858351804316044, -0.020571179687976837, 0.008488629013299942, -0.012102666310966015, -0.029797367751598358, 0.019484957680106163, 0.011579670011997223, 0.0070872679352760315, 0.0011214243713766336, -0.015032785944640636, -0.013349810615181923, -0.006912935990840197, 0.008186900988221169, -0.011418748646974564, 0.0207857433706522, 0.23430225253105164, -0.018063481897115707, 0.018787629902362823, 0.024889251217246056, 0.002029962372034788, 0.001196018303744495, 0.007127498276531696, 0.01434886734932661, -0.011888103559613228, -0.02046389877796173, 0.009608377702534199, 0.008133260533213615, -0.042429737746715546, 1.6474587027914822e-05, 0.01012466847896576, -0.019820211455225945, -0.04631868004798889, -0.053828369826078415, -0.024379665032029152, 0.009179252199828625, 0.014697532169520855, -0.0161324180662632, 0.007429226767271757, -0.04776697978377342, 0.020933255553245544, -0.0001175484067061916, -0.0025278145913034678, -0.01163331139832735, 0.01712477020919323, 0.020477309823036194, -0.023078879341483116, 0.004492402542382479, 0.006158614531159401, 0.018291454762220383, -0.026485059410333633, 0.0003817703982349485, 0.014040433801710606, -0.001401361427269876, 0.01720523089170456, 0.0052534290589392185, 0.009172547608613968, -0.008039388805627823, 0.005608798004686832, 0.009366994723677635, -0.0043515958823263645, 0.006520688533782959, -0.022301090881228447, -0.006594444625079632, -0.012913980521261692, 0.021831735968589783, -0.018130531534552574, -0.017017489299178123, 0.015502141788601875, 0.018640117719769478, 0.0044320570304989815, 0.02169763296842575, -0.012652482837438583, 0.030789721757173538, 0.03376677632331848, 0.019404496997594833, -0.025680450722575188, 0.012129486538469791, -0.008495334535837173, 0.017674585804343224, -0.0038017795886844397, -0.003932528663426638, -0.010788471437990665, 0.021711044013500214, 0.0022512301802635193, -0.032988984137773514, 0.0047304327599704266, -0.03379359468817711, -0.013376631774008274, -0.0004555262567009777, -0.038353048264980316, -0.022086529061198235, 0.02374938689172268, 0.011070084758102894, 0.023333672434091568, 0.03457138314843178, -0.005793187767267227, 0.0009965422796085477, 0.010701305232942104, -0.02738353982567787, -0.007724250666797161, -0.04318070411682129, -0.004455524496734142, -0.006034570746123791, -0.01819758303463459, -0.023601876571774483, 0.020396849140524864, -0.027008056640625, -0.011029853485524654, -0.00127396488096565, -0.008904344402253628, -0.000632372684776783, 0.0243260245770216, 0.0028446295764297247, 0.009480981156229973, 0.0007174434140324593, -0.022368142381310463, 0.030575159937143326, 0.04945665970444679, 0.008461808785796165, -0.017111359164118767, 0.0027809313032776117, 9.104239143198356e-05, -0.0026820313651114702, 0.0035805117804557085, -0.01650790311396122, -0.0072683049365878105, -0.011666836217045784, 0.014751172624528408, 0.00076437892857939, -0.009809529408812523, 0.027893126010894775, -0.01786232925951481, 0.021684223785996437, 0.019873851910233498, -0.022797266021370888, -0.01198197528719902, -0.018586477264761925, -0.012592136859893799, 0.01918993517756462, -0.0021942369639873505, -0.014643890783190727, 0.01021183468401432, 0.005337242502719164, -0.021791504696011543, 0.002745729638263583, 0.017889149487018585, -0.024339433759450912, 0.01630675047636032, -0.004727080464363098, -0.013369926251471043, -0.02631072700023651, -0.0030206379014998674, 0.017634356394410133, 0.007013511843979359, 0.025586578994989395, -0.0026434771716594696, -0.012189832516014576, 0.0029334716964513063, 0.00046851736260578036, 0.01700407825410366, -0.021563531830906868, 0.007301830220967531, -0.0028932413551956415, 0.02243519201874733, -0.010573908686637878, -0.005732841789722443, 0.018023250624537468, 0.0007828178931958973, -0.02502335235476494, 0.016320161521434784, -0.013202299363911152, -0.031594328582286835, -0.041893329471349716, 0.010533678345382214, -0.006664847955107689, -0.028965938836336136, 0.004710317589342594, 0.005830065812915564, -0.0037146133836358786, -0.011391928419470787, -0.0179025586694479, -0.1699335128068924, -0.00815337523818016, 0.02939506433904171, -0.03607332333922386, 0.026914184913039207, -0.01750025525689125, 0.008649551309645176, -0.005243371240794659, -0.016279930248856544, -0.015756934881210327, 0.012665892951190472, -0.01873398944735527, -0.015823984518647194, -0.02993147075176239, -0.021456250920891762, 0.004737137816846371, -0.024433305487036705, 0.004294602666050196, 0.0005347299738787115, 0.008341117762029171, 0.03457138314843178, -0.024634458124637604, 0.004636561498045921, -0.001847249106504023, -0.020812563598155975, 0.00815337523818016, -0.0008565737516619265, 0.025412246584892273, 0.0031580920331180096, -0.021174637600779533, 0.008931164629757404, 0.009031740948557854, 0.028509993106126785, -0.009427339769899845, 0.0026853838935494423, 0.004629856441169977, 0.018331684172153473, -0.038969915360212326, -0.010875636711716652, 0.01853283680975437, 0.026029113680124283, -0.007925402373075485, 0.012478150427341461, -0.007650495041161776, -0.031889352947473526, 0.011023148894309998, 0.021711044013500214, -0.0035235187970101833, -0.00033923506271094084, -0.03486640751361847, 0.01564965210855007, -0.0014432681491598487, 0.009353584609925747, 0.013799051754176617, 0.016494492068886757, 0.0003191198338754475, 0.009299944154918194, 0.01913629285991192, 0.008723307400941849, -0.01897537149488926, -0.021389199420809746, -0.006765424273908138, -0.002408799482509494, 0.007355471141636372, -0.019954312592744827, -0.009688838385045528, -0.010942688211798668, -0.01621287874877453, -0.030441056936979294, -0.0021171285770833492, -0.03384723514318466, -0.0021959131117910147, -0.03494687005877495, -0.024875840172171593, 0.030199673026800156, 0.0075298030860722065, -0.015274169854819775, -0.007107383105903864, 0.002097013173624873, 0.002348453737795353, -0.011318172328174114, 0.020597999915480614, -0.009092085994780064, 0.018465785309672356, 0.009668722748756409, -0.01031241100281477, 0.01163331139832735, -0.007409111596643925, -0.05157546326518059, -0.00897139497101307, 0.009782709181308746, -0.02400417998433113, -0.0007346251513808966, -0.016320161521434784, -0.0005598740535788238, 0.017017489299178123, 0.026860544458031654, -0.0188144501298666, 0.025465887039899826, -0.012424509972333908, 0.007147613447159529, 0.01438909862190485, -0.01045992225408554, 0.001652801875025034, 0.030870182439684868, 0.01737956330180168, -0.028268611058592796, 0.0028396006673574448, 0.02362869679927826, -0.008495334535837173, -0.012954211793839931, -0.003349186619743705, 0.023481184616684914, 0.01927039586007595, -0.018492605537176132, 0.04664052650332451, 0.0035101084504276514, -0.00758344354107976, 0.02601570449769497, 0.018881501629948616, 0.059970222413539886, 0.015528962016105652, -0.027812665328383446, 0.0021456251852214336, -0.016454262658953667, -0.04503130540251732, -0.09011625498533249, -0.018492605537176132, 0.011707066558301449, -0.0038218949921429157, -0.005669143982231617, 0.0042007314041256905, -0.02042366936802864, 0.010158193297684193, -0.031057924032211304, 0.013155363500118256, -0.03242575749754906, -0.002286431845277548, 0.025707270950078964, -0.004814246203750372, 0.011063379235565662, -0.0005623884499073029, 0.0005242532934062183, -0.00921948254108429, -0.017057718709111214, 0.029529165476560593, -0.017392972484230995, 0.0013904656516388059, 0.018224403262138367, 0.002210999606177211, -0.0059574623592197895, 0.004844419192522764, -0.03508096933364868, 0.01716500148177147, 0.002995493821799755, -0.013490617275238037, 0.0017232050886377692, -0.02000795304775238, 0.0068660001270473, -0.018506016582250595, 0.01823781244456768, 0.0015807222807779908, -0.01851942576467991, -0.020973484963178635, 0.004737137816846371, -0.026404598727822304, -0.02070528268814087, -0.010949392803013325, 0.0028245141729712486, 0.003677735570818186, -0.0024389722384512424, -0.01676269620656967, -0.01889491081237793, 0.014630480669438839, 0.012893865816295147, -0.040284112095832825, -0.01923016458749771, -0.01311513315886259, -0.01939108595252037, 0.00903844553977251, 0.04409259557723999, 0.00443540932610631, -0.010573908686637878, -0.0015471968799829483, 0.004807541146874428, -0.014710942283272743, -0.0033374526537954807, -0.012799995020031929, -0.02104053646326065, 0.0314602293074131, 0.03116520680487156, -0.01976657100021839, -0.0034397051203995943, -0.02148307114839554, 0.008984805084764957, -0.019042422994971275, -0.013182183727622032, 0.016253110021352768, -8.695648284628987e-05, 0.010573908686637878, -0.030441056936979294, -0.014456149190664291, -0.029421884566545486, -0.0061217364855110645, 0.0014826604165136814, -0.006098269019275904, -0.02547929808497429, -0.014563430100679398, -0.012370869517326355, -0.009715658612549305, -0.006027865689247847, -0.0058602383360266685, -0.021845145151019096, -0.004180616233497858, 0.022421782836318016, -0.02100030519068241, -0.0026434771716594696, 0.03172843158245087, 0.008160080760717392, -0.0063195363618433475, -0.003939233720302582, -0.0012823462020605803, 0.001398008898831904, 0.007865057326853275, 0.0018858032999560237, 0.049376197159290314, -0.0062021976336836815, -0.008864114060997963, -0.05723454803228378, 0.019337445497512817, -0.010111258365213871, -0.015448501333594322, -0.003422942478209734, 0.0094072250649333, 0.020021364092826843, 0.013470502570271492, 0.007382291369140148, -0.008582500740885735, -0.023910310119390488, -0.0029502345714718103, -0.02096007578074932, -0.0063027734868228436, -0.009494391269981861, -0.014724352397024632, 0.02551952749490738, -0.01835850439965725, 0.028509993106126785, 0.009601672179996967, 0.0035134609788656235, 0.00992351584136486, 0.008877524174749851, 0.03154068812727928, -0.00674195634201169, -0.023212980479002, 0.0062658959068357944, 0.0005238342564553022, 0.005719431675970554, 0.00612508924677968, 0.009232892654836178, -0.01737956330180168, 0.009313354268670082, 0.03170160949230194, 0.007630379870533943, 0.0034363525919616222, 0.008012568578124046, 0.003184912260621786, 0.016682235524058342, 0.03902355581521988, -0.021992657333612442, -0.027276258915662766, 0.016561543568968773, -0.020075004547834396, -0.00959496758878231, -0.0010937659535557032, 0.013611309230327606, -0.002539548324421048, 0.036582909524440765, -2.574645259301178e-05, 0.027651743963360786, 0.021509891375899315, -0.011385222896933556, -0.009769299067556858, -0.02116122655570507, 0.011519324965775013, 0.015998316928744316, 0.0022596113849431276, -0.00241382815875113, -0.0207857433706522, 0.03306944668292999, 0.005518279504030943, -0.0015731790335848927, 0.00585688604041934, 0.008562385104596615, -0.01056049857288599, -0.014429328963160515, -0.008401462808251381, 0.025667039677500725, -0.04041821137070656, -0.005843475926667452, -0.0035168135073035955, 0.014308637008070946, 0.010030797682702541, 0.027759024873375893, 0.004978520795702934, -0.016481082886457443, 0.0221535786986351, -0.025962064042687416, 0.0214428398758173, 0.0029519107192754745, -0.003681088099256158, -0.0008469352032989264, 0.00765720009803772, 0.025707270950078964, 0.0292341411113739, -0.008200311101973057, 0.0026904128026217222, -0.01074153557419777, 0.0009990567341446877, -0.0020634878892451525, 0.008448398672044277, 0.004244314506649971, 0.0008528021280653775, 0.011271236464381218, 0.018640117719769478, -0.00030172851984389126, 0.01082870177924633, 0.012323933653533459, -0.018506016582250595, 0.02030297741293907, 0.006235722918063402, -0.016950437799096107, -0.029958290979266167, -0.02478197030723095, 0.003966053947806358, -0.020879613235592842, -0.02679349295794964, 0.006379881873726845, 0.018841270357370377, -0.0025579873472452164, -0.01090916246175766, -0.013349810615181923, -0.007020216900855303, -0.02531837671995163, 0.014751172624528408, 0.007288420107215643, -0.031889352947473526, -0.026659391820430756, 0.005551804788410664, 0.03001193143427372, 0.015448501333594322, 0.04924209415912628, 0.01799643039703369, 0.02050413005053997, 0.023320263251662254, 0.0016100569628179073, 0.0034531154669821262, 0.028590453788638115, 0.002425562124699354, -0.01811712235212326, -0.002916709054261446, 0.02478197030723095, -0.007060447707772255, -0.026699621230363846, 0.01531440019607544, -0.012706123292446136, 0.03537599369883537, -0.0015136714791879058, 0.0840548649430275, 0.0010702981380745769, -0.0031681496184319258, -0.011680246330797672, -0.028295431286096573, -0.0035168135073035955, 0.006728546228259802, 0.020021364092826843, -0.006047980859875679, 0.02054435946047306, -0.019042422994971275, 0.013604603707790375, -0.018988782539963722, -0.021509891375899315, -0.00341623742133379, 0.0221535786986351, -0.0036375049967318773, -0.002668621251359582, -0.015140066854655743, 0.0031312715727835894, 0.04245655611157417, -0.010258769616484642, -0.009930221363902092, 0.0025730738416314125, -0.030521519482135773, -0.021429430693387985, 0.039291758090257645, 0.0029385006055235863, -0.0014365630922839046, -0.03660972788929939, 0.004073335323482752, 0.017138181254267693, -0.04757923632860184, -0.02342754416167736, 0.007556623313575983, 0.009762594476342201, -0.0069531663320958614, -0.0021741215605288744, 0.017687996849417686, 0.0015656357863917947, 0.0205845907330513, 0.01951177790760994, -0.005853533279150724, -0.0205845907330513, -0.022797266021370888, 0.014496379531919956, -0.016320161521434784, -0.028590453788638115, -0.02112099714577198], '_distance': 0.40166381001472473}),\n",
+ " Document(page_content='I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:', metadata={'vector': [-0.010849842801690102, -0.01362568698823452, 0.0007098066271282732, -0.01547191571444273, -0.00605549942702055, 0.019788449630141258, 0.006253774277865887, -0.022128738462924957, -0.02020450122654438, -0.0073394086211919785, 0.010472796857357025, 0.024195995181798935, 0.006585315335541964, -0.011252893134951591, 0.022778820246458054, 0.0045668152160942554, 0.014093744568526745, -0.006689328234642744, 0.00788547657430172, -0.017214130610227585, -0.017032109200954437, -0.006890852935612202, 0.005853974726051092, -0.04277529567480087, -0.014288769103586674, 0.0014269265811890364, 0.024924084544181824, -0.01536790281534195, -0.02052954211831093, 0.0002716117596719414, 0.027095353230834007, 0.0011774582089856267, 0.0013310398207977414, -0.02026950940489769, 0.016486041247844696, 0.0071378834545612335, 0.006465050391852856, 0.0008629818330518901, 0.00765144731849432, -0.03250402212142944, 0.02966967225074768, 0.0050771282985806465, 0.0018559797899797559, -0.023506909608840942, -0.0032910325098782778, 0.001704836031422019, -0.01770819164812565, -0.027199367061257362, 0.0026133235078305006, 0.02860354073345661, 0.012241015210747719, 0.017695190384984016, 0.0018120993627235293, -0.0007215893128886819, -0.0005875102360732853, 0.025275127962231636, -0.009237643331289291, 0.026003219187259674, 0.0038744795601814985, -0.005392417311668396, -0.026965338736772537, 0.0013286019675433636, -0.008672073483467102, 0.011681945994496346, -0.0036599531304091215, 0.001828351290896535, -0.003533187322318554, 0.0008304778020828962, -0.009257146157324314, -0.007833469659090042, 0.02658829092979431, 0.04155314341187477, -0.009556182660162449, 0.00412801094353199, 0.014210758730769157, -0.0069298576563596725, 0.002382545033469796, -0.004784592427313328, 0.0043555391021072865, -0.003299158299341798, -0.006734833586961031, -0.008457547053694725, -0.022063732147216797, 0.007976487278938293, 0.010635316371917725, 0.02725137397646904, 0.015887966379523277, -0.0032796559389680624, -0.020685561001300812, 0.001045004348270595, -0.00872408039867878, 0.010505300015211105, 0.016863087192177773, 0.03726261109113693, 0.00759944086894393, 0.010862844996154308, 0.005008870270103216, 0.014275766909122467, 0.008633068762719631, -0.02046453393995762, 0.0047813416458666325, -0.008444545790553093, -0.006799841765314341, -0.009991737082600594, -0.038900814950466156, -0.003559190547093749, 0.017747197300195694, -0.022440778091549873, 0.01294960267841816, -0.02462504804134369, -0.014470791444182396, 0.017149122431874275, -0.011499923653900623, -0.010661319829523563, 0.003585193771868944, -0.0274593997746706, 0.011733952909708023, -0.01070682518184185, 0.003832224290817976, -0.01625201106071472, 0.028421517461538315, 0.007963486015796661, 0.029045594856142998, -0.011077371425926685, 0.0004509933351073414, -0.028083477169275284, 0.005814970005303621, 0.008145508356392384, -0.010557306930422783, -0.011902973055839539, 0.02307785674929619, 0.01222151331603527, 0.0032455266918987036, -0.0003924860793631524, -0.013976730406284332, 0.03375217691063881, -0.02236276865005493, 0.005899480078369379, -0.0069298576563596725, -0.013859715312719345, 0.009738205932080746, 0.001019813702441752, -0.026341261342167854, -0.02751140482723713, -0.00010025459778262302, -0.0024979342706501484, 0.012254016473889351, 0.00256781792268157, 0.00832103006541729, -0.007079376373440027, 0.007501928601413965, -0.02007448486983776, -7.613052002852783e-05, -0.011012363247573376, 0.008600564673542976, 0.030969833955168724, 0.006312281358987093, 0.0015626309905201197, -0.001376545405946672, 0.00781396683305502, 0.01932039111852646, 0.006491053383797407, 0.019424404948949814, 0.003971991594880819, 0.0062927789986133575, 0.0025873202830553055, 0.019580423831939697, -0.012084996327757835, 0.0062862783670425415, -0.006231021136045456, -0.02099759876728058, -0.002237902022898197, -0.03840675577521324, 0.018566299229860306, 0.01717512682080269, 0.00436204019933939, -0.0006163575453683734, -0.005931984167546034, -0.014574804343283176, -0.028707554563879967, 0.031983960419893265, -0.011967981234192848, 0.030423766002058983, 0.01631701923906803, -0.02818748913705349, 0.004384792875498533, 0.011168382130563259, -0.014106745831668377, 0.0020818826742470264, -0.018280262127518654, 0.010713325813412666, 0.03744463622570038, 0.011129377409815788, -0.011090372689068317, -0.6511206030845642, -0.022388771176338196, 0.00905562099069357, -0.025522159412503242, -0.0030391262844204903, -0.008717578835785389, -0.008275524713099003, 0.0030001213308423758, -0.022115737199783325, 0.01625201106071472, 0.009712202474474907, -0.007969986647367477, -0.012689570896327496, 0.006799841765314341, 0.010030741803348064, -0.005220146384090185, -0.014717821963131428, -0.0335441529750824, 0.01804623380303383, 0.0065073054283857346, -0.002445927821099758, 0.011746954172849655, -0.043737415224313736, -0.016147999092936516, 0.006429295986890793, -0.011330902576446533, 0.01933339238166809, 0.011766456998884678, 0.011759955435991287, 0.0018007229082286358, -0.02268780767917633, -0.003955739550292492, 0.018410278484225273, -0.017409155145287514, 0.03562441095709801, 0.007521430961787701, -0.007742458488792181, 0.016564050689339638, -0.011311400681734085, 0.035754427313804626, -0.01311862375587225, -0.02557416632771492, 0.032113976776599884, -0.0008824842516332865, 0.01670706830918789, 0.017201129347085953, 0.023246876895427704, -0.0007837532903067768, -0.001135203056037426, 0.0011953354114666581, 0.01799422688782215, -0.00852905586361885, -2.3463842808268964e-05, 0.008054496720433235, 0.02315586619079113, -0.008932105265557766, 0.031229866668581963, 0.01043379120528698, 0.003910234197974205, 0.020490536466240883, -0.00437829177826643, 0.0007691264618188143, -0.007202891632914543, -0.030085723847150803, -0.024065978825092316, 0.0033316623885184526, -0.001347291748970747, -0.026731308549642563, 0.006045748479664326, -0.019554421305656433, 0.0023240377195179462, 0.013989731669425964, -0.023064855486154556, -0.0029237368144094944, 0.0032455266918987036, 0.020217502489686012, 0.003585193771868944, -0.01730514131486416, -0.0046513257548213005, 0.027485402300953865, 0.055542875081300735, -0.0014496794901788235, -0.01844928413629532, -0.032712049782276154, 0.035260364413261414, 0.004514808766543865, -0.032660044729709625, -0.014808833599090576, -0.00203150138258934, -0.005473677534610033, -0.0014545550802722573, 0.006270026322454214, -0.014743825420737267, -0.03479230776429176, -0.008132507093250751, 0.007670949678868055, 0.012793583795428276, -0.0021127616055309772, -0.0014797457261011004, -0.012761079706251621, 0.018878336995840073, -0.0020575046073645353, 0.007605941500514746, -0.010258269496262074, -0.009380660951137543, -0.020451530814170837, 0.0023142865393310785, 0.016330022364854813, 0.024208996444940567, -0.008314529433846474, -0.02276581898331642, -0.010043743066489697, -0.017474163323640823, -0.011493423022329807, -0.01980145089328289, -0.036950573325157166, 0.00508362939581275, 0.020308513194322586, -0.007865973748266697, -0.02162167616188526, 0.018553296104073524, 0.03193195164203644, -0.012579057365655899, -0.0101672587916255, 0.022778820246458054, 0.006035997066646814, -0.0019014853751286864, -0.024456027895212173, -0.014444787986576557, -0.011246392503380775, -0.0036079466808587313, -0.009211639873683453, 0.026198243722319603, -0.0012700947700068355, 0.006409793626517057, 0.021907711401581764, 0.012592058628797531, -0.018748320639133453, -0.014353777281939983, -0.005434672813862562, -0.005057625938206911, -0.004807345103472471, 0.01778620108962059, 0.01184446644037962, -0.03094383142888546, -0.02430000714957714, -0.00959518738090992, -0.005743461195379496, -0.01294960267841816, 0.006406542845070362, 0.0015585679793730378, -0.007254898082464933, -0.03195795789361, -0.006432546302676201, 0.014587805606424809, -0.01215000357478857, 0.012234514579176903, -0.012644065544009209, -0.014535799622535706, -0.01670706830918789, -0.005886478815227747, 0.03939487785100937, -0.009543181397020817, 0.016616057604551315, -0.018709316849708557, 0.0019031105330213904, 0.015744948759675026, 0.01731814444065094, -0.008464047685265541, -0.038562774658203125, -0.005210394971072674, -0.04012296721339226, -0.003021249081939459, 0.0012822836870327592, -0.01268307026475668, -0.009562683291733265, -0.026133235543966293, -0.026549287140369415, 0.006062000524252653, -0.00748242624104023, -0.01952841691672802, -0.003737962804734707, -0.025470152497291565, 0.007566936779767275, 0.045141588896512985, 0.02228475920855999, 0.029097601771354675, -0.0031561406794935465, -0.01578395441174507, 0.02000947669148445, -0.003310534870252013, -0.00030939767020754516, 0.015380904078483582, -0.01764318346977234, -0.013196633197367191, 0.014886843040585518, 0.008178012445569038, -0.008750082924962044, 0.016863087192177773, 0.014977853745222092, 0.03156790882349014, -0.0071313828229904175, 0.036534521728754044, -0.02704334817826748, 0.009510677307844162, -0.0021062607411295176, 0.009042619727551937, -0.03242601454257965, 0.008373036049306393, 0.0036859563551843166, -0.0005050312611274421, -0.038302741944789886, -0.0027108355425298214, -0.021569669246673584, 0.00429703202098608, 0.012754579074680805, 0.0075604356825351715, 0.015939973294734955, -0.01898234896361828, 0.005480178166180849, 0.015081866644322872, 0.002419924596324563, 0.013300646096467972, 0.010674321092665195, -0.05213645473122597, 0.01530289463698864, 0.022791821509599686, -0.009179136715829372, 0.00905562099069357, -0.006942859385162592, 0.016616057604551315, 0.014236762188374996, 0.028421517461538315, 0.02423500083386898, 0.01678507775068283, -0.009081624448299408, 0.0036274490412324667, -0.01154542900621891, 0.01899535208940506, 0.0123450281098485, 0.028629543259739876, 0.02489808201789856, 0.0038419757038354874, -0.006968862842768431, 0.02638026513159275, 0.03463628888130188, 0.00875658355653286, -0.006166013423353434, -0.04139712452888489, 0.02221975103020668, -0.0032113974448293447, 0.012481545098125935, 0.010154256597161293, -0.0023402897641062737, 0.012364530935883522, -0.004072754178196192, -0.00024357702932320535, 0.016069989651441574, 0.03461028262972832, 0.0060782525688409805, 0.014717821963131428, 0.003955739550292492, -0.010225765407085419, 0.0013562303502112627, 0.016395028680562973, -0.004638324026018381, -0.01161043718457222, -0.04142312705516815, -0.018709316849708557, 0.008353534154593945, -0.010030741803348064, 0.0006825844757258892, 0.005272152833640575, -0.04033099114894867, -0.0032390260603278875, -0.0014561802381649613, 0.016941096633672714, 0.006994865834712982, -0.013781705871224403, 0.004469303414225578, -0.0013838588492944837, -0.02987769804894924, -0.009751207195222378, -0.000403049896704033, 0.004329536110162735, -0.00896460935473442, 0.009822716005146503, -0.006669825874269009, -0.036482516676187515, 0.008711078204214573, 0.007963486015796661, 0.010108751244843006, 0.0081520089879632, -0.001049879938364029, -0.026458274573087692, 0.010323277674615383, 0.009523678570985794, -0.02370193414390087, 0.03133387863636017, 0.009185637347400188, 0.009166134521365166, 0.009913727641105652, -0.02758941613137722, -0.03263403847813606, 0.03840675577521324, 0.012546553276479244, -0.015744948759675026, -0.004755338653922081, -0.004917858634144068, -0.017019106075167656, -0.029357634484767914, 0.002247653203085065, -0.01738315261900425, -0.019814452156424522, 0.014418784528970718, -0.026198243722319603, 0.016017982736229897, 0.005551687441766262, 0.04727385193109512, 0.014483792707324028, -0.0020445031113922596, -0.018423279747366905, -0.0018299765652045608, 0.014210758730769157, 0.05705106258392334, 0.020243505015969276, -0.0015910719521343708, 0.027745435014367104, -0.017071112990379333, -0.010862844996154308, -0.014288769103586674, -0.0229608416557312, 0.006364287808537483, -0.00949767604470253, 2.392092937952839e-05, -0.00765144731849432, 0.027173364534974098, -0.021192623302340508, 0.013261641375720501, 0.015328897163271904, -0.0013513547601178288, -0.0014415534678846598, 0.00686484994366765, -0.03200996294617653, -0.010583310388028622, 0.006153011694550514, -0.008776086382567883, 0.028811566531658173, 0.041449129581451416, 0.02281782403588295, 0.038224730640649796, 0.025002095848321915, -0.0016690816264599562, -0.0254441499710083, -0.010635316371917725, 0.014873840846121311, -0.003949238918721676, 0.006520307157188654, -0.014925847761332989, 0.045869678258895874, 0.018345270305871964, 0.05273452773690224, -0.009504176676273346, -0.007280901074409485, -0.0031805187463760376, 0.0010685697197914124, 0.004492056090384722, -0.03213997930288315, 0.00676083704456687, -0.017552172765135765, -0.009166134521365166, 0.026861324906349182, 0.01665506139397621, -0.029903702437877655, 0.008477048948407173, 0.003052127780392766, -0.03437625616788864, 0.003614447545260191, 0.007781463209539652, -0.0008776086615398526, -0.005190892610698938, -0.0019274885999038815, -0.005213645286858082, -0.006617819424718618, -0.010004738345742226, -0.014080743305385113, 0.009471672587096691, -0.007241896353662014, -0.006773838307708502, -0.021842703223228455, -0.007742458488792181, 0.0013846714282408357, -0.01759117841720581, 0.01657705195248127, -0.0009946231730282307, -0.012923600152134895, -0.019502414390444756, 0.00209650956094265, 0.05203244090080261, 0.0026198243722319603, -0.0016674564685672522, -0.0058572250418365, 0.00808700080960989, 0.003432424971833825, -0.0020526291336864233, -0.02509310655295849, 0.017760198563337326, 0.006198517512530088, -0.004293781705200672, 0.0030651295091956854, -0.021257631480693817, 0.00736541161313653, -0.010355781763792038, 0.03450627252459526, 0.01335265301167965, -0.0033316623885184526, -0.005610194522887468, -0.016356024891138077, -0.0003819222911261022, 0.02886357344686985, -0.004791093058884144, 0.0071313828229904175, 0.006621069740504026, -0.017409155145287514, -0.006094504613429308, -0.02075056917965412, 0.009881223551928997, -0.02549615688621998, 0.005038123577833176, 0.024182993918657303, 0.008711078204214573, 0.01184446644037962, -0.0025304381269961596, -0.0012676569167524576, 0.024729061871767044, -6.36875702184625e-05, -0.0022053979337215424, -0.0011847716523334384, 0.006179014686495066, 0.015042861923575401, 0.017877213656902313, 0.0009288024739362299, -0.014522797428071499, -0.018501291051506996, 0.007716455031186342, -0.0354163832962513, 0.03401220962405205, 0.019762447103857994, -0.01933339238166809, 0.04430948570370674, -0.005873477086424828, -0.000269173935521394, -0.025158114731311798, 0.003533187322318554, -0.018137244507670403, 0.017877213656902313, -0.005047874990850687, 0.007969986647367477, -0.05658300593495369, 0.022635802626609802, -0.005798717960715294, -0.000987309729680419, -0.0022541540674865246, -0.022375769913196564, 0.0029838692862540483, -0.004102007951587439, -0.0012969105737283826, -0.020906588062644005, 0.015575927682220936, -0.035884443670511246, -0.01712311990559101, 0.02925362065434456, 0.025613170117139816, -0.005850724410265684, -0.02315586619079113, 0.005099881440401077, -0.03544238582253456, -0.012416536919772625, 0.008210516534745693, -0.03825073316693306, 0.0004883729270659387, -0.00029477087082341313, 0.012754579074680805, 0.03094383142888546, 0.013443663716316223, 0.009562683291733265, -0.001526063890196383, 0.013248640112578869, 0.0024670553393661976, -0.022193746641278267, 0.010466295294463634, 0.024143988266587257, -0.04022698104381561, 0.023480907082557678, 0.014782830141484737, 0.009107626974582672, -0.004238524474203587, -0.0013919847551733255, 0.04727385193109512, 0.01886533573269844, -0.002715711249038577, -0.011811962351202965, 0.009796712547540665, -0.034532275050878525, -0.003468179376795888, 0.01066782046109438, -0.0010831966064870358, 0.0119484793394804, -0.019632430747151375, 0.0021452654618769884, 0.02953965589404106, 0.0003049283695872873, 0.016460036858916283, 7.866989471949637e-05, 0.03159391134977341, -0.005122634116560221, 0.029825692996382713, -0.004696831572800875, -0.008106503635644913, -0.006994865834712982, -0.006552811246365309, -0.03874479606747627, 0.002566192764788866, 0.014886843040585518, 0.006039247382432222, 0.021244630217552185, -0.006832345854490995, 0.006786840036511421, 0.014756826683878899, -0.011532427743077278, 0.005480178166180849, -0.006608068011701107, -0.01217600703239441, -0.020100487396121025, -0.006400042213499546, -0.01952841691672802, -0.00875658355653286, 0.014743825420737267, 0.018904339522123337, -0.008347033523023129, -0.025340136140584946, 0.008854095824062824, -0.016291016712784767, -0.018033232539892197, 0.025249125435948372, -0.0003031000087503344, 0.02612023428082466, 0.027433395385742188, 0.026536284014582634, 0.001197773264721036, -0.002873355755582452, -0.01818925142288208, 0.025067102164030075, -0.029357634484767914, 0.008633068762719631, 0.0026750811375677586, 0.039290864020586014, -0.006166013423353434, -0.019424404948949814, 0.0037509643007069826, -0.008282025344669819, 0.0006062000175006688, -0.02470305748283863, 0.0019193625776097178, 0.027199367061257362, 0.018423279747366905, -0.014886843040585518, 0.01167544536292553, -0.016291016712784767, 0.033934202045202255, 0.024599045515060425, 0.027225369587540627, -0.01037528458982706, 0.005551687441766262, -0.020854581147432327, 0.01211750041693449, -0.008984112180769444, 0.01899535208940506, -0.014470791444182396, 0.0015480041038244963, -0.0030456269159913063, 0.007807466667145491, -0.003952489234507084, -0.004183267708867788, 0.005925483535975218, 0.03193195164203644, -0.008613565936684608, 0.003614447545260191, 0.025522159412503242, 0.0036339499056339264, -0.002939988858997822, -0.011421914212405682, -0.014301770366728306, 0.02275281585752964, -0.01164944190531969, -0.0028294753283262253, -0.033258117735385895, 0.004046750720590353, 0.015458913519978523, -0.02383195050060749, -0.0015512545360252261, 0.007189889904111624, -0.007696952670812607, -0.0119484793394804, 0.0060782525688409805, -0.0011530802585184574, -0.038432758301496506, -0.007872474379837513, 0.004982866812497377, -0.008073999546468258, 0.0021907712798565626, 0.005467176903039217, 0.00892560463398695, -0.020789572969079018, -0.01730514131486416, 9.916266208165325e-06, 0.018696313723921776, 0.02013949304819107, -0.01181846298277378, -0.017812205478549004, 0.008737081661820412, 0.028811566531658173, 0.01006324589252472, -0.006101005245000124, 0.02784944698214531, 0.006994865834712982, -0.029383637011051178, 0.03968091309070587, 0.005756462458521128, -0.023987969383597374, 0.006962361745536327, -0.013612684793770313, 0.0034779305569827557, -0.0055776904337108135, 0.010648318566381931, 0.009653694927692413, -0.006559311877936125, 0.023506909608840942, -0.012943102046847343, 0.001224589068442583, -0.0021290136501193047, 0.008587563410401344, -0.009088125079870224, 0.0025483155623078346, -0.006832345854490995, 0.00855505932122469, 0.004560314584523439, 0.004300282336771488, 0.006903854664415121, 0.011928976513445377, -0.01479583140462637, -0.0005338785704225302, -0.03086582012474537, 0.011688446626067162, -0.01238403283059597, -0.013950726948678493, 0.002151766326278448, -0.004615571349859238, 0.01691509410738945, 0.002372793620452285, -0.019541418179869652, 0.009374160319566727, -0.011304899118840694, 0.009250645525753498, 0.014873840846121311, 0.022648803889751434, 0.0012018362758681178, -0.00788547657430172, -0.027173364534974098, -0.006721831858158112, 0.0014708071248605847, -0.0016674564685672522, -0.02557416632771492, -0.005616695154458284, -0.004693580791354179, 0.009855220094323158, 0.002775843720883131, -0.020919589325785637, -0.02589920535683632, -0.015094868838787079, -0.03978492319583893, -0.004306782968342304, -0.004056502133607864, 0.013417661190032959, 0.012468543834984303, -0.008158509619534016, -0.0073459092527627945, 0.0369245707988739, 0.012455541640520096, 0.004163765348494053, -0.011681945994496346, -0.02007448486983776, -0.00245730415917933, 0.012839089147746563, -0.03364816680550575, -0.0071313828229904175, -0.028005465865135193, -0.030241742730140686, 0.024208996444940567, 0.013547676615417004, -0.01154542900621891, 0.03367416933178902, -0.009738205932080746, -0.002777468878775835, -0.006403292529284954, 0.000895485864020884, 0.03154190629720688, 0.02489808201789856, 0.010342780500650406, -0.038354746997356415, 0.007072875741869211, 0.0297996886074543, -0.029643669724464417, -0.022310761734843254, -0.010557306930422783, -0.0007069625426083803, 0.015523921698331833, -0.008691576309502125, -0.02228475920855999, 0.008327530696988106, 0.023597920313477516, -0.014405783265829086, 0.015445912256836891, 0.0005667888908647001, -0.0024979342706501484, 0.020295511931180954, 0.028681550174951553, -0.004917858634144068, -0.004739086609333754, 0.021127615123987198, -0.02899358980357647, -0.007846470922231674, 0.01318363193422556, -0.02201172523200512, 0.00736541161313653, -0.014275766909122467, -0.015757950022816658, -0.03515635058283806, 0.003386919153854251, -0.011616937816143036, 0.008808590471744537, -0.018267260864377022, 0.013599683530628681, 0.001076695742085576, -0.02852553129196167, -0.0068388464860618114, 0.01044029276818037, 0.0170581117272377, -0.02502809837460518, -0.02865554764866829, 0.006806342396885157, -0.0021972719114273787, 0.0060782525688409805, -0.008646070025861263, -0.006009994074702263, 0.01523788645863533, -0.013599683530628681, -0.0011319526238366961, 0.00382572365924716, -0.010349281132221222, 0.22778819501399994, -0.01205249223858118, 0.0004985304549336433, 0.019541418179869652, 0.00812600553035736, -0.0183192677795887, 0.027433395385742188, -0.008327530696988106, -0.007579938508570194, -0.013989731669425964, -0.0013830461539328098, 0.00741741806268692, -0.04277529567480087, -0.006465050391852856, 0.003351164748892188, -0.024677054956555367, -0.04139712452888489, -0.012273519299924374, -0.019307389855384827, 0.00032138352980837226, 0.009627691470086575, 0.004056502133607864, -0.00960168894380331, -0.022986846044659615, 0.012377532199025154, -0.006702329497784376, 0.00376721634529531, 0.0170581117272377, 0.01825425960123539, 0.014431786723434925, -0.014717821963131428, -0.0046448251232504845, 0.011824963614344597, 0.02605522610247135, -0.017552172765135765, -0.011467419564723969, 0.03203596547245979, -0.01127239502966404, 0.02704334817826748, 0.012982106767594814, 0.008496551774442196, -0.0026848323177546263, -0.011935477145016193, -0.0198664590716362, -0.02147865854203701, 0.025054100900888443, -0.012735076248645782, -0.003377167973667383, 0.0011132628424093127, 0.012650566175580025, -0.018683312460780144, 0.0005497243255376816, 0.00466432748362422, 0.030319754034280777, 0.006549560930579901, 0.013651689514517784, 0.021127615123987198, 0.020191499963402748, 0.03531236946582794, -0.0032780307810753584, -0.01181846298277378, 0.02751140482723713, -0.01739615388214588, 0.03081381507217884, -0.011915975250303745, -0.0026377015747129917, -0.01657705195248127, 0.008678574115037918, 0.011467419564723969, -0.01584896259009838, 0.006123757921159267, -0.00949767604470253, -0.014522797428071499, -0.007904978469014168, -0.045063577592372894, -0.014275766909122467, 0.0413191132247448, 0.04555764049291611, 0.02792745642364025, 0.01161043718457222, -0.01570594497025013, -0.003965490963310003, 0.025795193389058113, -0.034870315343141556, 0.009192137978971004, -0.02430000714957714, 0.007930981926620007, 0.016551049426198006, 0.002480057068169117, 0.010258269496262074, 0.007495427969843149, -0.031723927706480026, 0.010030741803348064, 0.01311862375587225, -0.00532090850174427, 0.017552172765135765, 0.01657705195248127, 0.011753454804420471, -0.0014464290579780936, -0.00026815818273462355, -0.009848719462752342, 0.015185879543423653, 0.024261003360152245, 0.03344013914465904, -0.03401220962405205, 0.009718703106045723, 0.01161043718457222, 0.008178012445569038, 0.016395028680562973, -0.0214136503636837, 0.009458671323955059, -0.0178512092679739, 0.013924723491072655, -0.009114128537476063, -0.016941096633672714, 0.0036664537619799376, -0.014171754010021687, 0.009978734888136387, -0.007709954399615526, -0.012026488780975342, -0.008776086382567883, -0.01906036026775837, 0.010277772322297096, 0.004596068989485502, -0.001463493681512773, -0.008828092366456985, -0.02093259058892727, -0.0020071235485374928, -0.003533187322318554, -0.002161517506465316, 0.025886204093694687, -0.009075123816728592, 0.019970472902059555, 0.0012725325068458915, -0.016603054478764534, -0.002481682226061821, 0.005681703332811594, 0.03679455444216728, 0.009627691470086575, 0.008282025344669819, -0.0016918344190344214, 0.0018754821503534913, -0.010934353806078434, -0.003167517017573118, 0.004800844471901655, -0.014990855939686298, -0.014496794901788235, -0.007898477837443352, -0.0007817217847332358, 0.0074304197914898396, -0.01500385720282793, 0.031177859753370285, 0.005730459466576576, -0.028941582888364792, 0.014951851218938828, -0.002208648482337594, -0.02966967225074768, -0.018696313723921776, -0.0012083370238542557, 0.01295610424131155, -0.03255603089928627, 0.012065493501722813, 0.017955223098397255, -0.0028538531623780727, 0.00031833627144806087, -0.019021354615688324, -0.16423633694648743, 0.00401099631562829, 0.04740386828780174, -0.030163733288645744, 0.03955089673399925, -0.005756462458521128, 0.046883802860975266, 0.008165011182427406, -0.0137296998873353, 0.009848719462752342, 0.00035774739808402956, -0.004462802316993475, -0.014444787986576557, -0.0024898082483559847, -0.018878336995840073, -0.02375394105911255, -0.012254016473889351, 0.012436039745807648, 0.032374005764722824, 0.015224884264171124, 0.03263403847813606, -0.038484763354063034, 0.004706582520157099, 0.0013424161588773131, -0.019918465986847878, 0.031983960419893265, 0.0079894894734025, 0.006877851206809282, 0.001549629378132522, -0.009010115638375282, -0.005847473628818989, -0.007157385814934969, 0.03739262744784355, -0.015588929876685143, 0.0010271271457895637, -0.009881223551928997, 0.0030960082076489925, -0.024065978825092316, 0.0003110228863079101, 0.011863968335092068, 0.022778820246458054, -0.0035754425916820765, 0.011590935289859772, -0.01127239502966404, -0.0036306993570178747, 0.002065630629658699, 0.011148880235850811, -0.003318660892546177, -0.0007289027562364936, -0.026731308549642563, 0.02704334817826748, -0.018761321902275085, -0.019619429484009743, 0.0032747804652899504, -0.002728712745010853, -0.019112365320324898, -0.016590053215622902, 0.02625024877488613, -0.001076695742085576, -0.021712686866521835, 0.015159877017140388, -0.00855505932122469, -0.012130501680076122, -0.009887724183499813, -0.011122876778244972, 0.002234651707112789, 0.0020542542915791273, 0.0034746802411973476, -0.0475078821182251, 0.003260153578594327, 0.008483550511300564, -0.010947355069220066, -0.011090372689068317, -0.009822716005146503, 0.0033284120727330446, 0.020568545907735825, -0.020828578621149063, -0.008568060584366322, 0.0134956706315279, -0.013963728211820126, 0.0011295147705823183, 0.033128101378679276, -0.02678331546485424, 0.02162167616188526, -0.002964366925880313, -0.003258528420701623, -0.01638202741742134, 0.00664382241666317, -0.03161991387605667, -0.013976730406284332, 0.03000771440565586, -0.018150247633457184, -0.03000771440565586, -0.011220389045774937, 0.030215740203857422, 0.018241258338093758, 0.020763570442795753, -0.017487164586782455, 0.008379537612199783, -0.0012814711080864072, -0.005766213871538639, 0.01570594497025013, -0.01833226904273033, -0.003770466661080718, 0.029903702437877655, 0.02531413361430168, 0.008828092366456985, 0.040773048996925354, 0.017682189121842384, 0.0002443896373733878, -0.03461028262972832, -0.009738205932080746, 0.013963728211820126, 0.024065978825092316, -0.01657705195248127, 0.01137640792876482, -0.004475804045796394, -0.00949767604470253, 0.019710440188646317, 0.02088058553636074, 0.024937087669968605, 0.0022314011584967375, -0.01980145089328289, -0.0028912329580634832, -0.02147865854203701, -0.022180745378136635, -0.10994160920381546, -0.036534521728754044, 0.017617180943489075, 0.017760198563337326, -0.005041373893618584, 0.013963728211820126, -0.00021919900609645993, -0.0028229744639247656, -0.027147360146045685, 0.012429538182914257, -0.014171754010021687, -0.028915580362081528, 0.014821834862232208, -0.011857467703521252, -0.008535556495189667, -0.01389872096478939, 0.014249764382839203, -0.004492056090384722, -0.021725689992308617, 0.02987769804894924, -0.007729456759989262, 0.003325161524116993, 0.003287781961262226, 0.020087486132979393, -0.007462923880666494, -0.005593942478299141, -0.018033232539892197, 0.008171511813998222, 0.009939730167388916, -0.013820710591971874, 0.00027628420502878726, -0.012000485323369503, 0.012813085690140724, -0.035546399652957916, 0.00281809875741601, -0.002868480049073696, -0.02054254338145256, -0.007976487278938293, 0.026497280225157738, -0.0022606549318879843, -0.03770466893911362, -0.012546553276479244, -0.024729061871767044, -0.03060578927397728, -0.031775932759046555, -0.010765332728624344, -0.03060578927397728, 0.02362392470240593, 0.0030066221952438354, -0.04683179780840874, 0.002660454250872135, 0.006094504613429308, -0.02797946333885193, 0.0006309843738563359, 0.03877079859375954, -0.013313648290932178, -0.013404658995568752, -0.024404020980000496, -0.013040614314377308, 0.0005830409354530275, 0.0018689814023673534, -0.025223122909665108, -0.023857953026890755, 0.03377818316221237, 0.019697438925504684, -0.023597920313477516, -0.018878336995840073, -0.025197118520736694, 0.01778620108962059, -0.023584919050335884, 0.0178512092679739, -0.005395668093115091, -0.015939973294734955, 0.02610723115503788, -0.029357634484767914, -0.011018863879144192, -0.01578395441174507, -0.009913727641105652, 0.03840675577521324, -0.004339287057518959, -0.005720708053559065, -0.01731814444065094, 0.017227131873369217, -0.03206196799874306, 0.01838427595794201, -0.0028538531623780727, 0.008347033523023129, 0.004573316313326359, 0.017201129347085953, -0.043737415224313736, -0.0005119383567944169, 0.02625024877488613, 0.004992618225514889, -0.014951851218938828, 0.007241896353662014, -0.0031301374547183514, -0.021114613860845566, -0.007371912710368633, 0.012377532199025154, 0.041449129581451416, -0.004973115399479866, -0.01345666591078043, -0.03367416933178902, 0.008243020623922348, -0.0065073054283857346, -0.005392417311668396, -0.0022655304055660963, -0.02328588254749775, 0.0032699047587811947, -0.008230018429458141, -0.006682827137410641, 0.0064162942580878735, -0.006062000524252653, 0.018761321902275085, -0.0046578263863921165, -0.012442540377378464, -0.00976420845836401, -0.01369069516658783, 0.00782046839594841, 0.006825844757258892, -0.003017998533323407, 0.0020786323584616184, -0.01920337788760662, 0.002304535359144211, 0.015536922961473465, 0.016421033069491386, -0.010251768864691257, 0.02504109963774681, -0.0007837532903067768, -0.0037119595799595118, -0.0029806189704686403, -0.005980740301311016, 0.014704820699989796, -0.024195995181798935, -0.010459794662892818, 0.04355539008975029, 0.00020924465206917375, -0.02202472649514675, -0.004765090066939592, 0.019853457808494568, 0.011727451346814632, 0.04025298357009888, 0.004407545551657677, -0.024065978825092316, 0.015887966379523277, -0.013950726948678493, -0.011246392503380775, -0.015744948759675026, -0.009894224815070629, 0.011421914212405682, 0.003203271422535181, 0.011935477145016193, 0.020763570442795753, 0.012299522757530212, 0.00045871303882449865, -0.01778620108962059, -0.0025548161938786507, -0.005535435397177935, 0.009244143962860107, 0.015380904078483582, -0.011974481865763664, -0.027225369587540627, 0.01912536658346653, 0.005889729131013155, 0.01010225061327219, -0.012202010490000248, 0.007839970290660858, -0.0023451652377843857, -0.0029676174744963646, -0.013768704608082771, 0.016421033069491386, -0.024339012801647186, -0.01770819164812565, 0.001675582374446094, 0.02335089072585106, 0.022167744114995003, 0.0015098118456080556, 0.001851104199886322, -0.021062606945633888, 0.0004132073954679072, -0.018176250159740448, 0.035260364413261414, 0.03409022092819214, 0.006013244390487671, -0.015523921698331833, 0.023532913997769356, 0.022102735936641693, 0.005376165732741356, -0.004173516761511564, 0.002668580273166299, -0.002813223283737898, -0.0020184998866170645, 0.006370788440108299, -0.0036957075353711843, 0.004589568357914686, -0.01534189935773611, -0.0006224520620889962, 0.021530665457248688, -0.00849005114287138, -0.0009239268838427961, 0.005298155825585127, 0.0206595566123724, -0.0008467297884635627, 0.009627691470086575, 0.011915975250303745, -0.04053901880979538, -0.03242601454257965, -0.003286156803369522, -0.006039247382432222, -0.03825073316693306, -0.0114804208278656, 0.0170581117272377, 4.553102553472854e-05, -0.01765618473291397, 0.006448798347264528, 0.011064369231462479, -0.015276891179382801, 0.005558188073337078, 0.004518059082329273, -0.013677692972123623, -0.030891824513673782, 0.019749443978071213, 0.0035429385025054216, 0.02860354073345661, 0.017500165849924088, -0.01322263665497303, 0.004823597148060799, 0.01244904100894928, 0.012663567438721657, 0.012293022125959396, 0.029721679165959358, 0.0031187611166387796, -0.015250887721776962, 0.0001362121693091467, -0.01231902465224266, -0.007183389272540808, -0.018371274694800377, -0.022258754819631577, 0.004319784697145224, 0.02115361951291561, -0.0031512652058154345, 0.09298750758171082, 0.03328412026166916, -0.037574652582407, 0.006621069740504026, -0.014132749289274216, 0.012904097326099873, 0.000840228982269764, 0.01352167408913374, -0.001222963910549879, -0.008776086382567883, 0.01534189935773611, 0.0067933411337435246, -0.022804822772741318, -0.028681550174951553, -0.010966857895255089, 0.015484916977584362, -0.0015796954976394773, -0.0010953856399282813, -0.0027595916762948036, -0.00792448129504919, 0.020698562264442444, -0.02504109963774681, 0.02228475920855999, 0.022323762997984886, -0.018280262127518654, -0.021725689992308617, 0.03159391134977341, 0.0050673773512244225, -0.00788547657430172, -0.01792921870946884, -0.009634193032979965, -0.0012928475625813007, -0.055542875081300735, -0.009114128537476063, 0.010661319829523563, -0.008366535417735577, 0.006682827137410641, -0.02966967225074768, 0.009270147420465946, 0.0014919346431270242, 0.014990855939686298, 0.00876308511942625, -0.015952974557876587, -0.03333612531423569, -0.030709801241755486, 0.03812072053551674, 0.012202010490000248, -0.01779920421540737, 0.0016308893682435155], '_distance': 0.41414153575897217})],\n",
+ " 'question': 'What is Vision transformer and How it is different from normal '\n",
+ " 'transformers?'}\n",
+ "'------------------------'\n",
+ "***** DOCS RELEVANCE CHECK *****\n",
+ "***** RATED DOCUMENT: NOT RELEVANT *****\n",
+ "***** RATED DOCUMENT: NOT RELEVANT *****\n",
+ "***** RATED DOCUMENT: NOT RELEVANT *****\n",
+ "***** RATED DOCUMENT: NOT RELEVANT *****\n",
+ "{ 'documents': [],\n",
+ " 'question': 'What is Vision transformer and How it is different from normal '\n",
+ " 'transformers?',\n",
+ " 'run_web_search': 'Yes'}\n",
+ "'------------------------'\n",
+ "***** DECIDE TO GENERATE *****\n",
+ "***** DECISION: TRANSFORM QUERY and RUN WEB SEARCH *****\n",
+ "***** TRANSFORM QUERY *****\n",
+ "{ 'documents': [],\n",
+ " 'question': 'What distinguishes a Vision Transformer from traditional '\n",
+ " 'Transformer models in machine learning?'}\n",
+ "'------------------------'\n",
+ "***** WEB SEARCH *****\n",
+ "{ 'documents': [ Document(page_content=\"Deep Learning System 2\\nAt the 2019 Neural Information Processing Systems Conference (NeurIPS 2019), Yoshua Bengio, one of the three pioneers of deep learning, gave a keynote speech that shed light on the possible transition of System 1 deep learning to System 2.\\nTwo systems of thinking. Integrating Deep Learning with other technologies\\nExtended Natural Language Processing Capabilities\\nIncreased Attention to Ethical Considerations\\nIntegration of Hybrid Models\\nSuch deep learning trends in 2024 will drive the scalability, popularity, and creation of more effective approaches in the field. If you want to build a technology product using deep learning, Merehead is your best choice!\\nSelf-supervised Learning\\nSelf-supervised learning is a paradigm in which a model learns to extract meaningful representations or features from unlabeled data without the need for explicit labels provided by humans.\\n Deep learning models successfully solve tasks such as:\\nDeep learning trends in 2024 may increase the number of significant advances and breakthroughs due to the availability of big data, computational resources, and advanced algorithms.\\n How have deep learning trends shaped?\\nUnderstanding how deep learning trends have formed in the past can give us a better idea of what we can expect from this field in the future.\\nTrain\\nDevelop\\nDeploy\\nOperate\\nData Collection\\nBuilding Blocks\\u200b\\nDevice Enrollment\\nMonitoring Dashboards\\nVideo Annotation\\u200b\\nApplication Editor\\u200b\\nDevice Management\\nRemote Maintenance\\nModel Training\\nApplication Library\\nDeployment Manager\\nUnified Security Center\\nAI Model Library\\nConfiguration Manager\\nIoT Edge Gateway\\nPrivacy-preserving AI\\nReady to get started?\\nVision Transformers (ViT) in Image Recognition – 2024 Guide\\nViso Suite is the all-in-one solution for teams to build, deliver, scale computer vision applications.\\n The self-attention layer calculates attention weights for each pixel in the image based on its relationship with all other pixels, while the feed-forward layer applies a non-linear transformation to the output of the self-attention layer. The final output of the ViT architecture is a class prediction, obtained by passing the output of the last transformer block through a classification head, which typically consists of a single fully connected layer.\\n The model also learns from training data to encode the relative location of the image patches to reconstruct the structure of the image.\\n The brighter the color of a pixel in the heatmap, the higher the attention weight between the corresponding tokens.\\nAfter their initial success in natural language processing, transformer ar-chitectures have rapidly gained traction in computer vision, providing state-of-the-art results for tasks such as image classification, detection, segmenta-tion, and video analysis.\\nThere are multiple blocks in the ViT encoder, and each block consists of three major processing elements:\\nLayer Norm keeps the training process on track and lets the model adapt to the variations among the training images.\\n The (single-head) attention mechanism in the above figure uses Q and K. Still, in the multi-head attention mechanism, each head has its projection matrix W_i^Q, W_i^K, and W_i^V, and they calculate the attention weights using the feature values projected using these matrices.\\n ViT vs. Convolutional Neural Networks\\nThe ViT model represents an input image as a series of image patches, like the series of word embeddings used when using transformers to text, and directly predicts class labels for the image.\\n Transformer Architecture modified for images (ViT)\\n\\u200d\\nThe paper suggests using a Transformer Encoder as a base model to extract features from the image and passing these “processed” features into a Multilayer Perceptron (MLP) head model for classification.\\n Although transformer-based models are known to only be effective when large training datasets are available, we show how we can effectively regularise the model during training and leverage pretrained image models to be able to train on comparatively small datasets\\nAutonomous driving\\nOn Tesla AI Day in 2021, Tesla revealed many intricate inner workings of the neural network powering Tesla FSD.\\nThere has been a dramatic increase in the complexity of the computer vision model landscape. Many models are now at your fingertips, from the first ConvNets to the latest Vision Transformers. Just as supervised learning on ImageNet gave way to self-supervised learning and image-text pair training, such as CLIP, so did training paradigms. Compared to ResNets, CLIP's visual encoder is far more ...\")],\n",
+ " 'question': 'What distinguishes a Vision Transformer from traditional '\n",
+ " 'Transformer models in machine learning?'}\n",
+ "'------------------------'\n",
+ "***** GENERATE *****\n",
+ "{ 'documents': [ Document(page_content=\"Deep Learning System 2\\nAt the 2019 Neural Information Processing Systems Conference (NeurIPS 2019), Yoshua Bengio, one of the three pioneers of deep learning, gave a keynote speech that shed light on the possible transition of System 1 deep learning to System 2.\\nTwo systems of thinking. Integrating Deep Learning with other technologies\\nExtended Natural Language Processing Capabilities\\nIncreased Attention to Ethical Considerations\\nIntegration of Hybrid Models\\nSuch deep learning trends in 2024 will drive the scalability, popularity, and creation of more effective approaches in the field. If you want to build a technology product using deep learning, Merehead is your best choice!\\nSelf-supervised Learning\\nSelf-supervised learning is a paradigm in which a model learns to extract meaningful representations or features from unlabeled data without the need for explicit labels provided by humans.\\n Deep learning models successfully solve tasks such as:\\nDeep learning trends in 2024 may increase the number of significant advances and breakthroughs due to the availability of big data, computational resources, and advanced algorithms.\\n How have deep learning trends shaped?\\nUnderstanding how deep learning trends have formed in the past can give us a better idea of what we can expect from this field in the future.\\nTrain\\nDevelop\\nDeploy\\nOperate\\nData Collection\\nBuilding Blocks\\u200b\\nDevice Enrollment\\nMonitoring Dashboards\\nVideo Annotation\\u200b\\nApplication Editor\\u200b\\nDevice Management\\nRemote Maintenance\\nModel Training\\nApplication Library\\nDeployment Manager\\nUnified Security Center\\nAI Model Library\\nConfiguration Manager\\nIoT Edge Gateway\\nPrivacy-preserving AI\\nReady to get started?\\nVision Transformers (ViT) in Image Recognition – 2024 Guide\\nViso Suite is the all-in-one solution for teams to build, deliver, scale computer vision applications.\\n The self-attention layer calculates attention weights for each pixel in the image based on its relationship with all other pixels, while the feed-forward layer applies a non-linear transformation to the output of the self-attention layer. The final output of the ViT architecture is a class prediction, obtained by passing the output of the last transformer block through a classification head, which typically consists of a single fully connected layer.\\n The model also learns from training data to encode the relative location of the image patches to reconstruct the structure of the image.\\n The brighter the color of a pixel in the heatmap, the higher the attention weight between the corresponding tokens.\\nAfter their initial success in natural language processing, transformer ar-chitectures have rapidly gained traction in computer vision, providing state-of-the-art results for tasks such as image classification, detection, segmenta-tion, and video analysis.\\nThere are multiple blocks in the ViT encoder, and each block consists of three major processing elements:\\nLayer Norm keeps the training process on track and lets the model adapt to the variations among the training images.\\n The (single-head) attention mechanism in the above figure uses Q and K. Still, in the multi-head attention mechanism, each head has its projection matrix W_i^Q, W_i^K, and W_i^V, and they calculate the attention weights using the feature values projected using these matrices.\\n ViT vs. Convolutional Neural Networks\\nThe ViT model represents an input image as a series of image patches, like the series of word embeddings used when using transformers to text, and directly predicts class labels for the image.\\n Transformer Architecture modified for images (ViT)\\n\\u200d\\nThe paper suggests using a Transformer Encoder as a base model to extract features from the image and passing these “processed” features into a Multilayer Perceptron (MLP) head model for classification.\\n Although transformer-based models are known to only be effective when large training datasets are available, we show how we can effectively regularise the model during training and leverage pretrained image models to be able to train on comparatively small datasets\\nAutonomous driving\\nOn Tesla AI Day in 2021, Tesla revealed many intricate inner workings of the neural network powering Tesla FSD.\\nThere has been a dramatic increase in the complexity of the computer vision model landscape. Many models are now at your fingertips, from the first ConvNets to the latest Vision Transformers. Just as supervised learning on ImageNet gave way to self-supervised learning and image-text pair training, such as CLIP, so did training paradigms. Compared to ResNets, CLIP's visual encoder is far more ...\")],\n",
+ " 'generation': 'A Vision Transformer (ViT) distinguishes itself from '\n",
+ " 'traditional Transformer models by applying the transformer '\n",
+ " 'architecture to image recognition tasks, where it treats an '\n",
+ " 'input image as a series of image patches similar to how text '\n",
+ " 'transformers treat a series of word embeddings. Unlike '\n",
+ " 'traditional models that might rely on convolutional layers, '\n",
+ " 'ViTs use self-attention mechanisms to calculate attention '\n",
+ " 'weights for each pixel in relation to all other pixels, '\n",
+ " 'enabling it to directly predict class labels for the image. '\n",
+ " 'This approach allows for the encoding of the relative '\n",
+ " \"location of image patches to reconstruct the image's \"\n",
+ " \"structure, leveraging the transformer's strengths in handling \"\n",
+ " 'sequential data for computer vision tasks.',\n",
+ " 'question': 'What distinguishes a Vision Transformer from traditional '\n",
+ " 'Transformer models in machine learning?'}\n",
+ "'------------------------'\n",
+ "{ 'documents': [ Document(page_content=\"Deep Learning System 2\\nAt the 2019 Neural Information Processing Systems Conference (NeurIPS 2019), Yoshua Bengio, one of the three pioneers of deep learning, gave a keynote speech that shed light on the possible transition of System 1 deep learning to System 2.\\nTwo systems of thinking. Integrating Deep Learning with other technologies\\nExtended Natural Language Processing Capabilities\\nIncreased Attention to Ethical Considerations\\nIntegration of Hybrid Models\\nSuch deep learning trends in 2024 will drive the scalability, popularity, and creation of more effective approaches in the field. If you want to build a technology product using deep learning, Merehead is your best choice!\\nSelf-supervised Learning\\nSelf-supervised learning is a paradigm in which a model learns to extract meaningful representations or features from unlabeled data without the need for explicit labels provided by humans.\\n Deep learning models successfully solve tasks such as:\\nDeep learning trends in 2024 may increase the number of significant advances and breakthroughs due to the availability of big data, computational resources, and advanced algorithms.\\n How have deep learning trends shaped?\\nUnderstanding how deep learning trends have formed in the past can give us a better idea of what we can expect from this field in the future.\\nTrain\\nDevelop\\nDeploy\\nOperate\\nData Collection\\nBuilding Blocks\\u200b\\nDevice Enrollment\\nMonitoring Dashboards\\nVideo Annotation\\u200b\\nApplication Editor\\u200b\\nDevice Management\\nRemote Maintenance\\nModel Training\\nApplication Library\\nDeployment Manager\\nUnified Security Center\\nAI Model Library\\nConfiguration Manager\\nIoT Edge Gateway\\nPrivacy-preserving AI\\nReady to get started?\\nVision Transformers (ViT) in Image Recognition – 2024 Guide\\nViso Suite is the all-in-one solution for teams to build, deliver, scale computer vision applications.\\n The self-attention layer calculates attention weights for each pixel in the image based on its relationship with all other pixels, while the feed-forward layer applies a non-linear transformation to the output of the self-attention layer. The final output of the ViT architecture is a class prediction, obtained by passing the output of the last transformer block through a classification head, which typically consists of a single fully connected layer.\\n The model also learns from training data to encode the relative location of the image patches to reconstruct the structure of the image.\\n The brighter the color of a pixel in the heatmap, the higher the attention weight between the corresponding tokens.\\nAfter their initial success in natural language processing, transformer ar-chitectures have rapidly gained traction in computer vision, providing state-of-the-art results for tasks such as image classification, detection, segmenta-tion, and video analysis.\\nThere are multiple blocks in the ViT encoder, and each block consists of three major processing elements:\\nLayer Norm keeps the training process on track and lets the model adapt to the variations among the training images.\\n The (single-head) attention mechanism in the above figure uses Q and K. Still, in the multi-head attention mechanism, each head has its projection matrix W_i^Q, W_i^K, and W_i^V, and they calculate the attention weights using the feature values projected using these matrices.\\n ViT vs. Convolutional Neural Networks\\nThe ViT model represents an input image as a series of image patches, like the series of word embeddings used when using transformers to text, and directly predicts class labels for the image.\\n Transformer Architecture modified for images (ViT)\\n\\u200d\\nThe paper suggests using a Transformer Encoder as a base model to extract features from the image and passing these “processed” features into a Multilayer Perceptron (MLP) head model for classification.\\n Although transformer-based models are known to only be effective when large training datasets are available, we show how we can effectively regularise the model during training and leverage pretrained image models to be able to train on comparatively small datasets\\nAutonomous driving\\nOn Tesla AI Day in 2021, Tesla revealed many intricate inner workings of the neural network powering Tesla FSD.\\nThere has been a dramatic increase in the complexity of the computer vision model landscape. Many models are now at your fingertips, from the first ConvNets to the latest Vision Transformers. Just as supervised learning on ImageNet gave way to self-supervised learning and image-text pair training, such as CLIP, so did training paradigms. Compared to ResNets, CLIP's visual encoder is far more ...\")],\n",
+ " 'generation': 'A Vision Transformer (ViT) distinguishes itself from '\n",
+ " 'traditional Transformer models by applying the transformer '\n",
+ " 'architecture to image recognition tasks, where it treats an '\n",
+ " 'input image as a series of image patches similar to how text '\n",
+ " 'transformers treat a series of word embeddings. Unlike '\n",
+ " 'traditional models that might rely on convolutional layers, '\n",
+ " 'ViTs use self-attention mechanisms to calculate attention '\n",
+ " 'weights for each pixel in relation to all other pixels, '\n",
+ " 'enabling it to directly predict class labels for the image. '\n",
+ " 'This approach allows for the encoding of the relative '\n",
+ " \"location of image patches to reconstruct the image's \"\n",
+ " \"structure, leveraging the transformer's strengths in handling \"\n",
+ " 'sequential data for computer vision tasks.',\n",
+ " 'question': 'What distinguishes a Vision Transformer from traditional '\n",
+ " 'Transformer models in machine learning?'}\n",
+ "'------------------------'\n",
+ "***** Generated Answer *****\n",
+ "('A Vision Transformer (ViT) distinguishes itself from traditional Transformer '\n",
+ " 'models by applying the transformer architecture to image recognition tasks, '\n",
+ " 'where it treats an input image as a series of image patches similar to how '\n",
+ " 'text transformers treat a series of word embeddings. Unlike traditional '\n",
+ " 'models that might rely on convolutional layers, ViTs use self-attention '\n",
+ " 'mechanisms to calculate attention weights for each pixel in relation to all '\n",
+ " 'other pixels, enabling it to directly predict class labels for the image. '\n",
+ " 'This approach allows for the encoding of the relative location of image '\n",
+ " \"patches to reconstruct the image's structure, leveraging the transformer's \"\n",
+ " 'strengths in handling sequential data for computer vision tasks.')\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Correction for question not present in context\n",
+ "inputs = {\n",
+ " \"keys\": {\n",
+ " \"question\": \"What is Vision transformer and How it is different from traditional transformers?\"\n",
+ " }\n",
+ "}\n",
+ "for output in app.stream(inputs):\n",
+ " for key, value in output.items():\n",
+ " # Node\n",
+ " # print full state\n",
+ " pprint.pprint(value[\"keys\"], indent=2, width=80, depth=None)\n",
+ " pprint.pprint(\"------------------------\")\n",
+ "\n",
+ "# Final generation\n",
+ "print(\"*\" * 5, \" Generated Answer \", \"*\" * 5)\n",
+ "pprint.pprint(value[\"keys\"][\"generation\"])"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/tutorials/Corrective-RAG-with_Langgraph/README.md b/tutorials/Corrective-RAG-with_Langgraph/README.md
new file mode 100644
index 0000000..98e23e9
--- /dev/null
+++ b/tutorials/Corrective-RAG-with_Langgraph/README.md
@@ -0,0 +1,29 @@
+[](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/Corrective-RAG-with_Langgraph/CRAG_with_Langgraph.ipynb)
+
+## Corrective RAG with Langgraph
+
+Self-reflection can enhance RAG, enabling correction of poor quality retrieval or generations.
+
+**Corrective Retrieval-Augmented Generation (CRAG)** is a method that works like a built-in fact-checker.
+
+It adds both creativity and accuracy by creating text and then checking for any mistakes or made-up information. This helps make sure the final result is reliable and matches real-world facts. It's like a safety feature for AI writers, making their work more trustworthy and lowering the chances of spreading false information.
+
+
+
+
+
+Corrective-RAG (CRAG) is a recent [paper]((https://arxiv.org/pdf/2401.15884.pdf)) that talks about a cool way to make a self-reflective RAG.
+
+The method givesating/scores retrieved documents based on how well they answer a question:
+
+For Correct documents -
+
+1. If at least one document is really relevant, it moves on to creating text
+2. Before creating text, it cleans up the knowledge
+3. This breaks down the document into "knowledge strips"
+4. It rates each strip and gets rid of ones that don't matter
+
+For Ambiguous or Incorrect documents -
+1. If all documents are not relevant enough or if it's not sure, the method looks for more information
+2. It uses a web search to add more details to what it found
+3. The diagram in the paper also show that they might change the question to get better results.
\ No newline at end of file