Can I Use a Custom LLM with OpenAI API Format? How to Set Base URL?

[lijiajun1997]

I am currently working on a project that involves custom LLM and I would like to know if it is possible to use my own LLM while adhering to the OpenAI API format.

If it is indeed possible, could you please guide me on where I can set the base URL for my custom LLM? I want to ensure that the API calls are directed to the correct endpoint.

[StreetLamb]

Hi @lijiajun1997 , Tribe does not support custom LLMs yet, but I can look into whether it’s a viable option. Could you provide more details on how you are serving the LLM and whether it supports tool calling? Tool calling is a key requirement for it to function properly on Tribe.

[lijiajun1997]

Hi @lijiajun1997 , Tribe does not support custom LLMs yet, but I can look into whether it’s a viable option. Could you provide more details on how you are serving the LLM and whether it supports tool calling? Tool calling is a key requirement for it to function properly on Tribe.

Thank you for your reply! I am managing my API keys through one-api, which follows the same format as the OpenAI API which supports tool calling; I just need to set a custom base URL using a viable option.

[StreetLamb]

@lijiajun1997 Please check out the latest release to enable using custom llm models. I have not tested it with one-api though. You can find more information in the readme.