Models and code to process data for I Love Conference.
Can be repurposed to create models and process data for any semantic search project.
- OpenAI API account: https://platform.openai.com/
- Pinecone account: https://www.pinecone.io/
Install rtx: https://github.com/jdx/rtx
- when you cd to the project directory, run
rtx installto install the correct version of python, poetry, and pipx
Install dependencies using poetry: poetry install
Install nox: poetry run pipx install nox && poetry run pipx inject nox nox-poetry
Install pre-commit as a git hook: poetry run pre-commit install
Install spacy model: poetry run python -m spacy download en_core_web_sm
Create a .env file with the following variables:
OPENAI_KEY=your_openai_api_key (found on API keys page)
OPENAI_ORG=your_openai_org (found on Settings page)
PINECONE_KEY=your_pinecone_api_key (found on API keys page)
PINECONE_ENV=your_pinecone_environment_name (found on API keys page)mkdir data
aws s3 sync s3://scripturecentralqa.data data
Activate the poetry virtual environment: poetry shell
Periodically add the files you are working on to git and run nox to run all checks and tests as you develop.
If nox fails, you can run the individual checks and tests manually; e.g., nox -s pre-commit, nox -s mypy-3.10, or nox -s tests-3.10
Run nox before creating a pull request to ensure that all checks pass.
After running poetry shell, you need to install the poetry virtual environment as a jupyter kernel.
Let's name it "models": python -m ipykernel install --user --name models
You only need to do this once.
You can run notebooks either in VS Code, or in your browser. To run notebooks in the browser, you run
env PYTHONPATH=`pwd` jupyter notebook
or (if you have fish shell)
env PYTHONPATH=(pwd) jupyter notebook
docker run -it -p 8080:8080 -v ~/iloveconference/labelstudio-data:/label-studio/data heartexlabs/label-studio:latest
Contributions are very welcome. To learn more, see the Contributor Guide.
Distributed under the terms of the MIT license, Models is free and open source software.
If you encounter any problems, please file an issue along with a detailed description.
This project was generated from @cjolowicz's Hypermodern Python Cookiecutter template.