Awesome Open Source AI

Awesome Open Source AI is a curated list of useful open-source AI resources, including inference engines, LLMs, and tools. Specifically, this list aims to link to resources that meet the Open Source Initiative® (OSI) definition of open source as much as possible.

Table of Contents

• Inference Engines
• Language Models
• Embedding Models
• Image Models
• Tools

Inference Engines

• llama.cpp - Barebones LLM inference with additional options to control LLM output beyond those of Ollama.
• Ollama - Easy-to-use LLM inference engine offering an OpenAI-compatible API server.
• OpenedAI Speech - Great OpenAI-compatible API TTS server that works with Piper TTS, which runs fast on the CPU.

Language Models

• Granite 3.1 - Great for RAG and suited for deployment on low-end consumer hardware.
• Qwen 2.5¹ - Best in-class language models that excel in benchmarks for math, etc.

Embedding Models

• all-MiniLM-L6-v2 - Great small embedding model perfectly suited for deployment on low-end consumer hardware.

Image Models

Coming soon...

Tools

• LLM Finetuning Script - A script for finetuning language models based on dataset from the synthetic dataset generation tool.
• Open WebUI - Provides a ChatGPT-like interface for chatting with local/external language models.
• Synthetic Dataset Generation w/ InternVL2 - A script for generating synthetic datasets from PDF files (using vision instead of text extraction).
• TextCraft - Add-in for Microsoft Word with tools for proofreading, text generation, etc.

Footnotes

1. The 3B and 72B models are not open-source. ↩