[Contribution Request] Integrate Unify AI into AgentLite

[KatoStevenMubiru]

Hello SalesforceAIResearch team,

I'm Kato Steven Mubiru, an AI engineer contributing to Unify AI. I'm reaching out to explore the possibility of integrating Unify into AgentLite

Before proceeding with a detailed proposal, I'd like to inquire about:

Your current process for accepting external contributions, especially for significant integrations.
Any specific guidelines or preferences for proposing and implementing new features.
The best way to discuss the potential benefits and implementation details of this integration.
Our goal would be to enhance AgentLite with Unify's capabilities.

https://unify.ai

I'd be happy to provide more details about Unify and our integration ideas once I understand your preferred process for such contributions.

Thank you for your time and guidance.

Best regards,
Kato Steven Mubiru
Unify AI

[JimSalesforce]

Hi Kato,

Thanks for your contribution! You could submit your PR and you will be prompt to sign the agreement of salesforce Contributor License. Since we want to keep the core code as simple as possible, it would be better that you could follow agentlite/llm designing for unify LLM inference. Also, it would be helpful if adding one or more tutorial notebooks regarding how to use unify for LLM inference.

Best,
Zhiwei
Salesforce AI Research

[KatoStevenMubiru]

Hi @JimSalesforce,

Apologies for the late reply!

Thanks for the feedback and guidance. I saw your points and we are currently working on addressing them. I'll make a pull request once we have implemented the changes and created the tutorial notebooks.

Thanks again!

[KatoStevenMubiru]

Hi @JimSalesforce ,

Following up on this issue, I've made good progress with the UnifyAI integration. I've created the UnifyLLM class, updated the LLMConfig to handle UnifyAI settings, and written comprehensive tests. The changes are in this branch: https://github.com/KatoStevenMubiru/AgentLite/tree/fix-issue-33.

However, I've encountered a challenge. AgentLite seems to have a hard dependency on the OpenAI API. Even when configuring the agent to use UnifyAI, AgentLite makes direct calls to the OpenAI API, resulting in an openai.AuthenticationError.

After careful analysis and debugging, I've traced the issue to the llm_layer function in agentlite/agents/BaseAgent.py. This function is always assuming an OpenAI-compatible LLM and doesn't have a way to differentiate between providers.

To resolve this, I believe we need to modify the BaseAgent class to be provider-aware. This would involve passing the llm_config to llm_layer and adding a provider check to conditionally use either the UnifyAI client or the OpenAI client.

Before I proceed with these changes to BaseAgent.py, I'd appreciate your guidance. Would this be an acceptable approach to support alternative LLM providers like UnifyAI within AgentLite's architecture?

Thanks!

Kato

[JimSalesforce]

Hi Kato, would that be possible on just defining a new subclass of BaseLLM? I would not suggest changing BaseAgent, which will lead to too many dependencies changed.

[KatoStevenMubiru]

Okay @JimSalesforce .Sure, I can try and work with a new subclass of the BaseLLM class.
I will let you know in case of anything.

Thank you