Add function calling support for models that support it.

[vkameswaran]

⚠️ Please check that this feature request hasn't been suggested before.

• I searched previous Ideas in Discussions didn't find any similar feature requests.
• I searched previous Issues didn't find any similar feature requests.

🔖 Feature description

I want us to use the LiteLLM function calling spec for instead of our hybrid GPT approach for models that support function calling. Cohere, Anthropic, Fireworks etc. etc.

Acknowledgements

• My issue title is concise and descriptive.
• I have searched the existing issues to make sure this feature has not been requested yet.
• I have provided enough information for the maintainers to understand and evaluate this request.

[greptile-apps-local]

To implement function calling support for models like LiteLLM, Cohere, Anthropic, and Fireworks, you'll need to adapt the existing infrastructure for LLM integration and custom tool creation. Here's a high-level approach based on the current codebase:

1. Extend LLM Provider Support: In libs/superagent/app/agents/langchain.py, add new classes for each of the new LLM providers that support function calling. Use ChatOpenAI and AzureChatOpenAI as references for how to structure these classes.

2. Adapt LLM Initialization: Modify the _get_llm method to initialize the correct LLM provider class based on the provider attribute of the LLM model. This might involve checking for support of function calling and choosing the appropriate class.

3. Custom Tool Integration: Utilize the guide in /fern/mdx/sdk/local_tools.mdx for creating custom tools. For models supporting function calling, ensure that the metadata for tools includes necessary information for function invocation.

4. Invoke Function Calls: In the agent's invocation logic, specifically in the method where the agent processes input and decides on actions, include logic to detect when a function call should be made to the LLM and execute it accordingly.

This approach leverages the existing infrastructure for LLM integration and custom tool functionality, extending it to accommodate the specific requirements of function calling with the mentioned models.

References

greptileai/superagent/fern/mdx/sdk/local_tools.mdx
greptileai/superagent/libs/superagent/app/agents/langchain.py

Ask Greptile