OpenAI tools agent error if use with gpt-4-turbo-2024-04-09

[os1ma]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain import hub
from langchain.agents import AgentExecutor, create_openai_tools_agent
from langchain_community.tools.tavily_search import TavilySearchResults
from langchain_openai import ChatOpenAI

tools = [TavilySearchResults(max_results=1)]

prompt = hub.pull("hwchase17/openai-tools-agent")
llm = ChatOpenAI(model="gpt-4-turbo-2024-04-09", temperature=0)
agent = create_openai_tools_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools)

result = agent_executor.invoke({"input": "What's the weather like in Tokyo?"})

print(result["output"])

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/src/langchain_agent.py", line 20, in <module>
    result = agent_executor.invoke({"input": "What's the weather like in Tokyo?"})
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain/chains/base.py", line 163, in invoke
    raise e
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain/chains/base.py", line 153, in invoke
    self._call(inputs, run_manager=run_manager)
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 1432, in _call
    next_step_output = self._take_next_step(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 1138, in _take_next_step
    [
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 1138, in <listcomp>
    [
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 1166, in _iter_next_step
    output = self.agent.plan(
             ^^^^^^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 514, in plan
    for chunk in self.runnable.stream(inputs, config={"callbacks": callbacks}):
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2875, in stream
    yield from self.transform(iter([input]), config, **kwargs)
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2862, in transform
    yield from self._transform_stream_with_config(
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1880, in _transform_stream_with_config
    chunk: Output = context.run(next, iterator)  # type: ignore
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2826, in _transform
    for output in final_pipeline:
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1283, in transform
    for chunk in input:
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 4722, in transform
    yield from self.bound.transform(
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1300, in transform
    yield from self.stream(final, config, **kwargs)
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 245, in stream
    raise e
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 225, in stream
    for chunk in self._stream(messages, stop=stop, **kwargs):
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/langchain_openai/chat_models/base.py", line 460, in _stream
    with self.client.create(messages=message_dicts, **params) as response:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/openai/_utils/_utils.py", line 275, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 667, in create
    return self._post(
           ^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 1213, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 902, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/home/oshima/work/src/os1ma/debug_langchain_gpt-4-turbo-2024-04-09/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 993, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Unknown parameter: 'messages[2].tool_calls[0].index'.", 'type': 'invalid_request_error', 'param': 'messages[2].tool_calls[0].index', 'code': 'unknown_parameter'}}

Description

OpenAI tools agent initialized by create_openai_tools_agent raise error if use with gpt-4-turbo-2024-04-09.
This error is not caused with other OpenAI model like gpt-4-0125-preview, gpt-3.5-turbo-0125.
If I only use openai package without langchain, this error is not caused.

System Info

$ python -m langchain_core.sys_info

System Information
------------------
> OS:  Linux
> OS Version:  #111~20.04.1-Ubuntu SMP Mon Mar 11 15:44:43 UTC 2024
> Python Version:  3.11.9 (main, Apr 10 2024, 18:31:06) [GCC 9.4.0]

Package Information
-------------------
> langchain_core: 0.1.41
> langchain: 0.1.15
> langchain_community: 0.0.32
> langsmith: 0.1.43
> langchain_openai: 0.1.2
> langchain_text_splitters: 0.0.1
> langchainhub: 0.1.15

Packages not installed (Not Necessarily a Problem)
--------------------------------------------------
The following packages were not found:

> langgraph
> langserve

[os1ma]

I understood this error cause, so I'm creating Pull Request.

[baskaryan]

fixed in #20271

[RadientBrain]

kindly check this bug when I upgraded langchain for using turbo: #20405