.with_structured_output throwing error with create_react_agent

[Sachin-Deloitte]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langgraph.prebuilt import create_react_agent
from langchain_core.pydantic_v1 import BaseModel, Field

class outputformat(BaseModel):
        AImessage: str = Field(description="Detailed answer of the user question")
        FinalSqlQuery: str = Field(description="Final SQL query which was executed")
        TableNames: str = Field(description="Name of all the tables accessed during to reach the final SQL query")

struct_llm = llm.with_structured_output(outputformat)

toolkit = SQLDatabaseToolkit(db=db, llm=llm)

tools = toolkit.get_tools()

agent_executor = create_react_agent(struct_llm, tools, state_modifier=system_message)

Description

I am trying to format the response of LLM using .with_structured_output however getting the below error:

in create_react_agent
model = model.bind_tools(tool_classes)

AttributeError: 'RunnableSequence' object has no attribute 'bind_tools'

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 23.5.0: Wed May 1 20:12:58 PDT 2024 RELEASE_ARM64_T6000
Python Version: 3.11.5 (v3.11.5, Aug 24 2023, 10:50:31) [Clang 13.0.0 (clang-1300.0.29.30)]

Package Information

langchain_core: 0.2.24
langchain: 0.2.11
langchain_community: 0.2.10
langsmith: 0.1.94
langchain_openai: 0.1.19
langchain_text_splitters: 0.2.2
langgraph: 0.1.17

[dosubot[bot]]

Hey @Sachin-Deloitte! I'm a bot here to help you with bug fixes, answer your questions, and guide you on becoming a contributor. Let's get that bug squashed! 🤖

The error AttributeError: 'RunnableSequence' object has no attribute 'bind_tools' occurs because the create_react_agent function expects a model that has the bind_tools method, but the RunnableSequence object does not have this method.

To resolve this, ensure that the model you pass to create_react_agent is compatible and has the bind_tools method. Here is an example of how you can modify your code to use a compatible model:

from langgraph.prebuilt import create_react_agent
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_openai import ChatOpenAI
from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit

class OutputFormat(BaseModel):
    AImessage: str = Field(description="Detailed answer of the user question")
    FinalSqlQuery: str = Field(description="Final SQL query which was executed")
    TableNames: str = Field(description="Name of all the tables accessed during to reach the final SQL query")

# Initialize the language model
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

# Use the `with_structured_output` method
struct_llm = llm.with_structured_output(OutputFormat)

# Initialize the SQLDatabaseToolkit
toolkit = SQLDatabaseToolkit(db=db, llm=llm)

# Get the tools from the toolkit
tools = toolkit.get_tools()

# Create the agent
agent_executor = create_react_agent(struct_llm, tools, state_modifier=system_message)

In this example, ChatOpenAI is used as the language model, which supports the bind_tools method. This should resolve the AttributeError and allow you to format the response of the LLM using .with_structured_output ^[1][2].

To continue talking to Dosu, mention @dosu.

[Sachin-Deloitte]

@dosu I am using Azure there the line of code is:
from langchain_openai import AzureChatOpenAI
from langgraph.prebuilt import create_react_agent
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_openai import ChatOpenAI
from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit

class OutputFormat(BaseModel):
AImessage: str = Field(description="Detailed answer of the user question")
FinalSqlQuery: str = Field(description="Final SQL query which was executed")
TableNames: str = Field(description="Name of all the tables accessed during to reach the final SQL query")
llm = AzureChatOpenAI(
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
azure_deployment=os.environ["AZURE_OPENAI_DEPLOYMENT_NAME"],
openai_api_version=os.environ["AZURE_OPENAI_API_VERSION"],
temperature=0,
max_tokens=100
)

    Use the `with_structured_output` method

struct_llm = llm.with_structured_output(OutputFormat)

Initialize the SQLDatabaseToolkit

toolkit = SQLDatabaseToolkit(db=db, llm=llm)

Get the tools from the toolkit

tools = toolkit.get_tools()

Create the agent

agent_executor = create_react_agent(struct_llm, tools, state_modifier=system_message)

Still getting same error :(

[dosubot[bot]]

The AzureChatOpenAI class does not support the bind_tools method, which is causing the AttributeError. To resolve this, you need to use a model that supports the bind_tools method. Unfortunately, AzureChatOpenAI does not have this method implemented.

You can use the ChatOpenAI class instead, which supports the bind_tools method. Here is how you can modify your code:

from langgraph.prebuilt import create_react_agent
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_openai import ChatOpenAI
from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit

class OutputFormat(BaseModel):
    AImessage: str = Field(description="Detailed answer of the user question")
    FinalSqlQuery: str = Field(description="Final SQL query which was executed")
    TableNames: str = Field(description="Name of all the tables accessed during to reach the final SQL query")

# Initialize the language model
llm = ChatOpenAI(model="gpt-4o", temperature=0)

# Use the `with_structured_output` method
struct_llm = llm.with_structured_output(OutputFormat)

# Initialize the SQLDatabaseToolkit
toolkit = SQLDatabaseToolkit(db=db, llm=llm)

# Get the tools from the toolkit
tools = toolkit.get_tools()

# Create the agent
agent_executor = create_react_agent(struct_llm, tools, state_modifier=system_message)

If you must use AzureChatOpenAI, you will need to implement a custom method to bind tools or use a different approach to integrate tools with your language model. Unfortunately, AzureChatOpenAI does not natively support the bind_tools method, which is required by the create_react_agent function ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[Sachin-Deloitte]

@dosu what are the different approach to integrate the tools with my language model, As I need to use AzureChatOpenAI

[dosubot[bot]]

To integrate tools with the AzureChatOpenAI model, you can use the bind method provided by the model. Here is an example of how you can bind a tool to the AzureChatOpenAI model:

import os
from langchain_openai import AzureChatOpenAI
from langchain_community.tools.convert_to_openai import format_tool_to_openai_function
from langchain.tools.retriever import create_retriever_tool
from langchain_core.retrievers import BaseRetriever
from langchain_core.documents import Document
from langchain_core.messages import AIMessage, HumanMessage
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.agents import AgentExecutor
from langchain.agents.output_parsers import OpenAIFunctionsAgentOutputParser
from langchain.callbacks.manager import CallbackManagerForRetrieverRun
from langchain_core.pydantic_v1 import BaseModel, Field
from typing import List, Tuple

class ArxivRetriever(BaseRetriever):
    """`Arxiv` retriever."""

    def _get_relevant_documents(
        self, query: str, *, run_manager: CallbackManagerForRetrieverRun
    ) -> List[Document]:
        # Implementation of the retriever
        pass

description = (
    "A wrapper around Arxiv.org "
    "Useful for when you need to answer questions about Physics, Mathematics, "
    "Computer Science, Quantitative Biology, Quantitative Finance, Statistics, "
    "Electrical Engineering, and Economics "
    "from scientific articles on arxiv.org. "
    "Input should be a search query."
)

# Create the tool
arxiv_tool = create_retriever_tool(ArxivRetriever(), "arxiv", description)
tools = [arxiv_tool]

# Initialize the AzureChatOpenAI model
llm = AzureChatOpenAI(
    temperature=0,
    azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    api_version=os.getenv("AZURE_OPENAI_API_VERSION"),
)

assistant_system_message = """You are a helpful research assistant. \
Lookup relevant information as needed."""

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", assistant_system_message),
        MessagesPlaceholder(variable_name="chat_history"),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

llm_with_tools = llm.bind(functions=[format_tool_to_openai_function(t) for t in tools])

def _format_chat_history(chat_history: List[Tuple[str, str]]):
    buffer = []
    for human, ai in chat_history:
        buffer.append(HumanMessage(content=human))
        buffer.append(AIMessage(content=ai))
    return buffer

agent = (
    {
        "input": lambda x: x["input"],
        "chat_history": lambda x: _format_chat_history(x["chat_history"]),
        "agent_scratchpad": lambda x: format_to_openai_function_messages(
            x["intermediate_steps"]
        ),
    }
    | prompt
    | llm_with_tools
    | OpenAIFunctionsAgentOutputParser()
)

class AgentInput(BaseModel):
    input: str
    chat_history: List[Tuple[str, str]] = Field(
        ..., extra={"widget": {"type": "chat", "input": "input", "output": "output"}}
    )

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True).with_types(
    input_type=AgentInput
)

In this example, the ArxivRetriever class is defined as a retriever tool, and it is then bound to the AzureChatOpenAI model using the bind method. This allows the model to use the tool during its operation. This approach is functionally equivalent to the bind_tools() method ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[bhupendra171]

@Sachin-Deloitte i am facing same issue. did you find a solution ?