Additional Resources

Meeseeks is usually tested to work well these tools and models.

OpenAI Compatible APIs 📚

• vllm-project/vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
• BerriAI/Lite-LLM: Call 100+ LLM APIs using the OpenAI API format.
  • My Lite-LLM configuration is available in the repository as an example.
• ollama/ollama: Get up and running with Llama 3, Mistral, Gemma, and other large language models locally.
• oobabooga/text-generation-webui: A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

My notes

• In some cases, TOOL_MODEL may require models with a larger context window. The number of tokens for prompts from the HomeAssistant tool is proportional to the number of entities or sensors accessible by Meeseeks.

Models

These are my subjective experiences, not an objective evaluation.

TODO: Create a validation dataset.

• microsoft/phi-3-mini-128k-instruct: It features large context windows, which is advantageous. Performance-wise, it delivers averagely for both ACTION_MODEL and TOOL_MODEL.
• Most of the Anthropic Claude 3 models demonstrate good performance. I suggest utilizing claude-3-haiku for both ACTION_MODEL and TOOL_MODEL as it strikes a commendable balance between quality, token cost, and latency.