Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LlamaCpp with mistral fails with [TypeError: can only concatenate str (not "NoneType") to str] when function calling. #1122

Open
5 tasks done
GabHoo opened this issue Jul 24, 2024 · 0 comments

Comments

@GabHoo
Copy link

GabHoo commented Jul 24, 2024

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangGraph/LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangGraph/LangChain rather than my code.
  • I am sure this is better as an issue rather than a GitHub discussion, since this is a LangGraph bug and not a design question.

Example Code

model_get_schema_sys="""You are an SQL engineer and you have to choose, given the user questions and the list of tables present in your datababase,
the most relevant tables that will be used in the query. You must select ONLY the tables names provided to you and make sure the spelling 
is correct. Call the tool and return ONLY the names of this tables separate by a comma to the tool.
Make sure the arguments are ONLY names from the available tables and double check the spelling.
"""
model_get_schema_prompt = ChatPromptTemplate.from_messages(
    [("system", model_get_schema_sys), ("placeholder", "{messages}")])


get_schema_bound_tool = model_get_schema_prompt | llm.bind_tools(
    [get_schema_tool],tool_choice=True)

###LETS TEST IT

big_openai_message={'messages': [HumanMessage(content='Which artist has the most songs after Iron Maiden?', id='639046f6-f4d0-4f4c-95da-0926243df500'), AIMessage(content='', id='73f957a5-6623-4cde-be02-a4f87b72989d', tool_calls=[{'name': 'sql_db_list_tables', 'args': {}, 'id': 'tool_abcd123', 'type': 'tool_call'}]), ToolMessage(content='Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track', name='sql_db_list_tables', id='935a7225-2719-482c-a032-be0f754893df', tool_call_id='tool_abcd123')]}

#Llamacpp+Mistral will NOT work with this
get_schema_bound_tool.invoke(big_openai_message)

#Llamacpp+Mistral WILL work with this. The message is just a mock message but it makes me think #is a formatting issue?
get_schema_bound_tool.invoke({"messages": [("user", "What is the artist with most tracks? Tables available on the Database are Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track")]})

Error Message and Stack Trace (if applicable)

4651 def invoke(
   4652     self,
   4653     input: Input,
   4654     config: Optional[RunnableConfig] = None,
   4655     **kwargs: Optional[Any],
   4656 ) -> Output:
-> 4657     return self.bound.invoke(
   4658         input,
   4659         self._merge_configs(config),
...
--> 939 raise rewrite_traceback_stack(source=source)

File <template>:2, in top-level template code()

TypeError: can only concatenate str (not "NoneType") to str

Description

I am trying to adapt your SQL langgraph Agent tutorial for a local model, using ChatLlamaCpp and Mistral (both support tool calling). It wouldn't work for that Type error so I troubled shot it until here. I added this prompt instead of just binding it right away. I dug out the state that is passed to the tool when using openAI model (in the code as big_openai_message) and try to test the tool with that but it fails. It does work with a mock message that I provided that is formatted differently. Any suggestions?

System Info

langgraph==0.1.9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant