Blog comment on Locally running RAG pipeline with Verba and Llama3 with Ollama

[giscus[bot]]

Blog comment on Locally running RAG pipeline with Verba and Llama3 with Ollama

Run your RAG pipelines entirely locally with Verba and Ollama.

https://weaviate.io/blog/local-llm-with-verba-for-rag

[levistamper]

Fresh rocky linux image, fresh install of python3.11

...errors immediately.

It will error out whether I setup .env file (with API keys, and URLs) or not

[stamper@172-233-194-249 ~]$ python3 -m venv golden
[stamper@172-233-194-249 ~]$ source golden/bin/activate
(golden) [stamper@172-233-194-249 ~]$ !pip
pip install goldenverba
Collecting goldenverba
Using cached goldenverba-1.0.4-py3-none-any.whl (9.8 MB)
Collecting weaviate-client==3.23.1
Using cached weaviate_client-3.23.1-py3-none-any.whl (108 kB)
Collecting python-dotenv==1.0.0
Using cached python_dotenv-1.0.0-py3-none-any.whl (19 kB)
Collecting openai==0.27.9
Using cached openai-0.27.9-py3-none-any.whl (75 kB)
Collecting wasabi==1.1.2
Using cached wasabi-1.1.2-py3-none-any.whl (27 kB)
Collecting fastapi==0.102.0
Using cached fastapi-0.102.0-py3-none-any.whl (66 kB)
Collecting uvicorn[standard]==0.29.0
Using cached uvicorn-0.29.0-py3-none-any.whl (60 kB)
Collecting click==8.1.7
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Collecting asyncio==3.4.3
Using cached asyncio-3.4.3-py3-none-any.whl (101 kB)
Collecting tiktoken==0.6.0
Using cached tiktoken-0.6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB)
Collecting requests==2.31.0
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Collecting pypdf==4.2.0
Using cached pypdf-4.2.0-py3-none-any.whl (290 kB)
Collecting python-docx==1.1.2
Using cached python_docx-1.1.2-py3-none-any.whl (244 kB)
Collecting pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4
Using cached pydantic-2.8.2-py3-none-any.whl (423 kB)
Collecting starlette<0.28.0,>=0.27.0
Using cached starlette-0.27.0-py3-none-any.whl (66 kB)
Collecting typing-extensions>=4.5.0
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Collecting tqdm
Using cached tqdm-4.66.4-py3-none-any.whl (78 kB)
Collecting aiohttp
Using cached aiohttp-3.9.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Collecting lxml>=3.1.0
Downloading lxml-5.2.2-cp311-cp311-manylinux_2_28_x86_64.whl (5.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.0/5.0 MB 86.2 MB/s eta 0:00:00
Collecting charset-normalizer<4,>=2
Using cached charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Collecting idna<4,>=2.5
Using cached idna-3.7-py3-none-any.whl (66 kB)
Collecting urllib3<3,>=1.21.1
Using cached urllib3-2.2.2-py3-none-any.whl (121 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2024.7.4-py3-none-any.whl (162 kB)
Collecting regex>=2022.1.18
Using cached regex-2024.7.24-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (786 kB)
Collecting h11>=0.8
Using cached h11-0.14.0-py3-none-any.whl (58 kB)
Collecting httptools>=0.5.0
Using cached httptools-0.6.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (318 kB)
Collecting pyyaml>=5.1
Downloading PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (757 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 757.7/757.7 kB 140.6 MB/s eta 0:00:00
Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0
Downloading uvloop-0.19.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.5/3.5 MB 97.1 MB/s eta 0:00:00
Collecting watchfiles>=0.13
Downloading watchfiles-0.22.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 145.3 MB/s eta 0:00:00
Collecting websockets>=10.4
Downloading websockets-12.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.9/130.9 kB 58.4 MB/s eta 0:00:00
Collecting validators<=0.21.0,>=0.18.2
Using cached validators-0.21.0-py3-none-any.whl (27 kB)
Collecting authlib>=1.1.0
Using cached Authlib-1.3.1-py2.py3-none-any.whl (223 kB)
Collecting cryptography
Using cached cryptography-43.0.0-cp39-abi3-manylinux_2_28_x86_64.whl (4.0 MB)
Collecting annotated-types>=0.4.0
Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB)
Collecting pydantic-core==2.20.1
Downloading pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 141.9 MB/s eta 0:00:00
Collecting anyio<5,>=3.4.0
Using cached anyio-4.4.0-py3-none-any.whl (86 kB)
Collecting aiosignal>=1.1.2
Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting attrs>=17.3.0
Downloading attrs-23.2.0-py3-none-any.whl (60 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 32.7 MB/s eta 0:00:00
Collecting frozenlist>=1.1.1
Downloading frozenlist-1.4.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (272 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 272.3/272.3 kB 91.9 MB/s eta 0:00:00
Collecting multidict<7.0,>=4.5
Downloading multidict-6.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (128 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.7/128.7 kB 51.3 MB/s eta 0:00:00
Collecting yarl<2.0,>=1.0
Downloading yarl-1.9.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (328 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 328.1/328.1 kB 96.7 MB/s eta 0:00:00
Collecting sniffio>=1.1
Using cached sniffio-1.3.1-py3-none-any.whl (10 kB)
Collecting cffi>=1.12
Downloading cffi-1.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (464 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 464.8/464.8 kB 115.5 MB/s eta 0:00:00
Collecting pycparser
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Installing collected packages: asyncio, websockets, wasabi, validators, uvloop, urllib3, typing-extensions, tqdm, sniffio, regex, pyyaml, python-dotenv, pypdf, pycparser, multidict, lxml, idna, httptools, h11, frozenlist, click, charset-normalizer, certifi, attrs, annotated-types, yarl, uvicorn, requests, python-docx, pydantic-core, cffi, anyio, aiosignal, watchfiles, tiktoken, starlette, pydantic, cryptography, aiohttp, openai, fastapi, authlib, weaviate-client, goldenverba
Successfully installed aiohttp-3.9.5 aiosignal-1.3.1 annotated-types-0.7.0 anyio-4.4.0 asyncio-3.4.3 attrs-23.2.0 authlib-1.3.1 certifi-2024.7.4 cffi-1.16.0 charset-normalizer-3.3.2 click-8.1.7 cryptography-43.0.0 fastapi-0.102.0 frozenlist-1.4.1 goldenverba-1.0.4 h11-0.14.0 httptools-0.6.1 idna-3.7 lxml-5.2.2 multidict-6.0.5 openai-0.27.9 pycparser-2.22 pydantic-2.8.2 pydantic-core-2.20.1 pypdf-4.2.0 python-docx-1.1.2 python-dotenv-1.0.0 pyyaml-6.0.1 regex-2024.7.24 requests-2.31.0 sniffio-1.3.1 starlette-0.27.0 tiktoken-0.6.0 tqdm-4.66.4 typing-extensions-4.12.2 urllib3-2.2.2 uvicorn-0.29.0 uvloop-0.19.0 validators-0.21.0 wasabi-1.1.2 watchfiles-0.22.0 weaviate-client-3.23.1 websockets-12.0 yarl-1.9.4

[notice] A new release of pip available: 22.3.1 -> 24.1.2
[notice] To update, run: pip install --upgrade pip

(golden) [stamper@172-233-194-249 ~]$ more /etc/os-release
NAME="Rocky Linux"
VERSION="9.4 (Blue Onyx)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.4"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Rocky Linux 9.4 (Blue Onyx)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:9::baseos"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2032-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-9"
ROCKY_SUPPORT_PRODUCT_VERSION="9.4"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.4"
(golden) [stamper@172-233-194-249 ~]$ python --version
Python 3.11.7
(golden) [stamper@172-233-194-249 ~]$ verba start
INFO: Will watch for changes in these directories: ['/home/stamper']
INFO: Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)
INFO: Started reloader process [15266] using WatchFiles
/home/stamper/golden/lib64/python3.11/site-packages/pypdf/_crypt_providers/_cryptography.py:32: CryptographyDeprecationWarning: ARC4 has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.ARC4 and will be removed from this module in 48.0.0.
from cryptography.hazmat.primitives.ciphers.algorithms import AES, ARC4
ℹ Setting up client
ℹ Using Weaviate Embedded
Started /home/stamper/.cache/weaviate-embedded: process ID 15271
{"action":"startup","default_vectorizer_module":"none","level":"info","msg":"the default vectorizer modules is set to "none", as a result all new schema classes without an explicit vectorizer setting, will use this vectorizer","time":"2024-07-26T22:53:11Z"}
{"action":"startup","auto_schema_enabled":true,"level":"info","msg":"auto schema enabled setting is set to "true"","time":"2024-07-26T22:53:11Z"}
{"action":"memberlist_init","bind_port":39223,"error":"Failed to get final advertise address: No private IP address found, and explicit IP not provided","hostname":"Embedded_at_6666","level":"error","msg":"memberlist not created","time":"2024-07-26T22:53:11Z"}
{"action":"startup","error":"create member list: Failed to get final advertise address: No private IP address found, and explicit IP not provided","level":"error","msg":"could not init cluster state","time":"2024-07-26T22:53:11Z"}
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/stamper/golden/lib64/python3.11/site-packages/uvicorn/_subprocess.py", line 78, in subprocess_started
target(sockets=sockets)
File "/home/stamper/golden/lib64/python3.11/site-packages/uvicorn/server.py", line 65, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
File "/home/stamper/golden/lib64/python3.11/site-packages/uvicorn/server.py", line 69, in serve
await self._serve(sockets)
File "/home/stamper/golden/lib64/python3.11/site-packages/uvicorn/server.py", line 76, in _serve
config.load()
File "/home/stamper/golden/lib64/python3.11/site-packages/uvicorn/config.py", line 433, in load
self.loaded_app = import_from_string(self.app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/stamper/golden/lib64/python3.11/site-packages/uvicorn/importer.py", line 19, in import_from_string
module = importlib.import_module(module_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1147, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/stamper/golden/lib64/python3.11/site-packages/goldenverba/server/api.py", line 37, in
manager = verba_manager.VerbaManager()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/stamper/golden/lib64/python3.11/site-packages/goldenverba/verba_manager.py", line 45, in init
self.client = self.setup_client()
^^^^^^^^^^^^^^^^^^^
File "/home/stamper/golden/lib64/python3.11/site-packages/goldenverba/verba_manager.py", line 209, in setup_client
client = weaviate.Client(
^^^^^^^^^^^^^^^^
File "/home/stamper/golden/lib64/python3.11/site-packages/weaviate/client.py", line 147, in init
url, embedded_db = self.__parse_url_and_embedded_db(url, embedded_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/stamper/golden/lib64/python3.11/site-packages/weaviate/client.py", line 288, in __parse_url_and_embedded_db
embedded_db.start()
File "/home/stamper/golden/lib64/python3.11/site-packages/weaviate/embedded.py", line 223, in start
self.wait_till_listening()
File "/home/stamper/golden/lib64/python3.11/site-packages/weaviate/embedded.py", line 168, in wait_till_listening
raise WeaviateStartUpError(
weaviate.exceptions.WeaviateStartUpError: Embedded DB did not start listening on port 6666 within 30 seconds
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='UTF-8'>