Skip to content

Latest commit

 

History

History
124 lines (112 loc) · 5.1 KB

README_MACOS.md

File metadata and controls

124 lines (112 loc) · 5.1 KB
Raw

macOS

Supports CPU and MPS (Metal M1/M2).

Install

  • Download and Install Miniconda for Python 3.10.
  • Run Miniconda
  • Setup environment with Conda Rust: 
    conda create -n h2ogpt python=3.10 rust
conda activate h2ogpt
  • Install dependencies: 
    git clone https://github.com/h2oai/h2ogpt.git
cd h2ogpt

# fix any bad env
pip uninstall -y pandoc pypandoc pypandoc-binary
pip install --upgrade pip
python -m pip install --upgrade setuptools

# Install Torch:
pip install -r requirements.txt --extra-index https://download.pytorch.org/whl/cpu -c reqs_optional/reqs_constraints.txt
  • Install document question-answer dependencies: 
    # Required for Doc Q/A: LangChain:
pip install -r reqs_optional/requirements_optional_langchain.txt -c reqs_optional/reqs_constraints.txt
# Required for CPU: LLaMa/GPT4All:
pip uninstall -y llama-cpp-python llama-cpp-python-cuda
pip install -r reqs_optional/requirements_optional_gpt4all.txt -c reqs_optional/reqs_constraints.txt
pip install librosa -c reqs_optional/reqs_constraints.txt
pip install llama-cpp-python -c reqs_optional/reqs_constraints.txt
# Optional: PyMuPDF/ArXiv:
pip install -r reqs_optional/requirements_optional_langchain.gpllike.txt -c reqs_optional/reqs_constraints.txt
# Optional: Selenium/PlayWright:
pip install -r reqs_optional/requirements_optional_langchain.urls.txt -c reqs_optional/reqs_constraints.txt
# Optional: DocTR OCR:
conda install weasyprint pygobject -c conda-forge -y
pip install -r reqs_optional/requirements_optional_doctr.txt -c reqs_optional/reqs_constraints.txt
# Optional: for supporting unstructured package
python -m nltk.downloader all
  • For supporting Word and Excel documents, download libreoffice: https://www.libreoffice.org/download/download-libreoffice/ .
  • To support OCR, install Tesseract Documentation: 
    brew install libmagic
brew link libmagic
brew install poppler
brew install tesseract
brew install tesseract-lang
brew install rubberband
brew install pygobject3 gtk4
brew install libjpeg
brew install libpng
brew install wget

See FAQ for how to run various models. See CPU and GPU for some other general aspects about using h2oGPT on CPU or GPU, such as which models to try.

Run

For information on how to run h2oGPT offline, see Offline.

In your terminal, run:

python generate.py --base_model=TheBloke/zephyr-7B-beta-GGUF --prompt_type=zephyr --max_seq_len=4096

Or you can run it from a file called run.sh that would contain following text:

#!/bin/bash
python generate.py --base_model=TheBloke/zephyr-7B-beta-GGUF --prompt_type=zephyr --max_seq_len=4096

and run sh run.sh from the terminal placed in the parent folder of run.sh

Issues

  • Metal M1/M2 Only: Verify whether torch uses MPS, run below python script:

     import torch
 if torch.backends.mps.is_available():
     mps_device = torch.device("mps")
     x = torch.ones(1, device=mps_device)
     print (x)
 else:
     print ("MPS device not found.")

    Output

    tensor([1.], device='mps:0')

  • If you see ld: library not found for -lSystem then ensure you do below and then retry from scratch to do pip install commands:

    export LDFLAGS=-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib`

  • If conda Rust has issus, you can download and install [Native Rust]((https://www.geeksforgeeks.org/how-to-install-rust-in-macos/):

    curl –proto ‘=https’ –tlsv1.2 -sSf https://sh.rustup.rs | sh
# enter new shell and test:
rustc --version

  • When running a Mac with Intel hardware (not M1), you may run into

    _clang: error: the clang compiler does not support '-march=native'_

    during pip install. If so, set your archflags during pip install. E.g.

    ARCHFLAGS="-arch x86_64" pip install -r requirements.txt -c reqs_optional/reqs_constraints.txt

  • Metal M1/M2 Only

  • If you encounter an error while building a wheel during the pip install process, you may need to install a C++ compiler on your computer.