-
Notifications
You must be signed in to change notification settings - Fork 13.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: add tool-calling agent #20328
Merged
Merged
docs: add tool-calling agent #20328
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
306 changes: 306 additions & 0 deletions
306
docs/docs/modules/agents/agent_types/tool_calling.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,306 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "raw", | ||
"metadata": {}, | ||
"source": [ | ||
"---\n", | ||
"sidebar_position: 0\n", | ||
"sidebar_label: Tool calling\n", | ||
"---" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"# Tool calling agent\n", | ||
"\n", | ||
"[Tool calling](/docs/modules/model_io/chat/function_calling) allows a model to detect when one or more tools should be called and respond with the inputs that should be passed to those tools. In an API call, you can describe tools and have the model intelligently choose to output a structured object like JSON containing arguments to call these tools. The goal of tools APIs is to more reliably return valid and useful tool calls than what can be done using a generic text completion or chat API.\n", | ||
"\n", | ||
"We can take advantage of this structured output, combined with the fact that you can bind multiple tools to a [tool calling chat model](/docs/integrations/chat/) and\n", | ||
"allow the model to choose which one to call, to create an agent that repeatedly calls tools and receives results until a query is resolved.\n", | ||
"\n", | ||
"This is a more generalized version of the [OpenAI tools agent](/docs/modules/agents/agent_types/openai_tools/), which was designed for OpenAI's specific style of\n", | ||
"tool calling. It uses LangChain's ToolCall interface to support a wider range of\n", | ||
"provider implementations, such as [Anthropic](/docs/integrations/chat/anthropic/), [Google Gemini](/docs/integrations/chat/google_vertex_ai_palm/), and [Mistral](/docs/integrations/chat/mistralai/)\n", | ||
"in addition to [OpenAI](/docs/integrations/chat/openai/).\n", | ||
"\n", | ||
"## Setup\n", | ||
"\n", | ||
"Any models that support tool calling can be used in this agent. [TODO ADD WHICH]\n", | ||
"\n", | ||
"This demo uses [Tavily](https://app.tavily.com), but you can also swap in any other [built-in tool](/docs/integrations/tools) or add [custom tools](/docs/modules/tools/custom_tools/).\n", | ||
"You'll need to sign up for an API key and set it as `process.env.TAVILY_API_KEY`.\n", | ||
"\n", | ||
"```{=mdx}\n", | ||
"import ChatModelTabs from \"@theme/ChatModelTabs\";\n", | ||
"\n", | ||
"<ChatModelTabs customVarName=\"llm\" />\n", | ||
"```" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 2, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"from langchain_anthropic import ChatAnthropic\n", | ||
"\n", | ||
"llm = ChatAnthropic(model=\"claude-3-sonnet-20240229\", temperature=0)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Initialize Tools\n", | ||
"\n", | ||
"We will first create a tool that can search the web:" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 3, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"from langchain.agents import AgentExecutor, create_tool_calling_agent\n", | ||
"from langchain_community.tools.tavily_search import TavilySearchResults\n", | ||
"from langchain_core.prompts import ChatPromptTemplate\n", | ||
"\n", | ||
"tools = [TavilySearchResults(max_results=1)]" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Create Agent\n", | ||
"\n", | ||
"Next, let's initialize our tool calling agent:" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 9, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"prompt = ChatPromptTemplate.from_messages(\n", | ||
" [\n", | ||
" (\n", | ||
" \"system\",\n", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This will fail for models that don't accept system messages (I think Vertex). Maybe leave out? |
||
" \"You are a helpful assistant. Make sure to use the tavily_search_results_json tool for information.\",\n", | ||
" ),\n", | ||
" (\"placeholder\", \"{chat_history}\"),\n", | ||
" (\"human\", \"{input}\"),\n", | ||
" (\"placeholder\", \"{agent_scratchpad}\"),\n", | ||
" ]\n", | ||
")\n", | ||
"\n", | ||
"# Construct the Tools agent\n", | ||
"agent = create_tool_calling_agent(llm, tools, prompt)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Run Agent\n", | ||
"\n", | ||
"Now, let's initialize the executor that will run our agent and invoke it!" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 10, | ||
"metadata": {}, | ||
"outputs": [ | ||
{ | ||
"name": "stdout", | ||
"output_type": "stream", | ||
"text": [ | ||
"\n", | ||
"\n", | ||
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n" | ||
] | ||
}, | ||
{ | ||
"name": "stderr", | ||
"output_type": "stream", | ||
"text": [ | ||
"/Users/bagatur/langchain/libs/partners/anthropic/langchain_anthropic/chat_models.py:347: UserWarning: stream: Tool use is not yet supported in streaming mode.\n", | ||
" warnings.warn(\"stream: Tool use is not yet supported in streaming mode.\")\n" | ||
] | ||
}, | ||
{ | ||
"name": "stdout", | ||
"output_type": "stream", | ||
"text": [ | ||
"\u001b[32;1m\u001b[1;3m\n", | ||
"Invoking: `tavily_search_results_json` with `{'query': 'LangChain'}`\n", | ||
"responded: [{'id': 'toolu_01QxrrT9srzkYCNyEZMDhGeg', 'input': {'query': 'LangChain'}, 'name': 'tavily_search_results_json', 'type': 'tool_use'}]\n", | ||
"\n", | ||
"\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://github.com/langchain-ai/langchain', 'content': 'About\\n⚡ Building applications with LLMs through composability ⚡\\nResources\\nLicense\\nCode of conduct\\nSecurity policy\\nStars\\nWatchers\\nForks\\nReleases\\n291\\nPackages\\n0\\nUsed by 39k\\nContributors\\n1,848\\nLanguages\\nFooter\\nFooter navigation Latest commit\\nGit stats\\nFiles\\nREADME.md\\n🦜️🔗 LangChain\\n⚡ Building applications with LLMs through composability ⚡\\nLooking for the JS/TS library? ⚡ Building applications with LLMs through composability ⚡\\nLicense\\nlangchain-ai/langchain\\nName already in use\\nUse Git or checkout with SVN using the web URL.\\n 📖 Documentation\\nPlease see here for full documentation, which includes:\\n💁 Contributing\\nAs an open-source project in a rapidly developing field, we are extremely open to contributions, whether it be in the form of a new feature, improved infrastructure, or better documentation.\\n What can you build with LangChain?\\n❓ Retrieval augmented generation\\n💬 Analyzing structured data\\n🤖 Chatbots\\nAnd much more!'}]\u001b[0m" | ||
] | ||
}, | ||
{ | ||
"name": "stderr", | ||
"output_type": "stream", | ||
"text": [ | ||
"/Users/bagatur/langchain/libs/partners/anthropic/langchain_anthropic/chat_models.py:347: UserWarning: stream: Tool use is not yet supported in streaming mode.\n", | ||
" warnings.warn(\"stream: Tool use is not yet supported in streaming mode.\")\n" | ||
] | ||
}, | ||
{ | ||
"name": "stdout", | ||
"output_type": "stream", | ||
"text": [ | ||
"\u001b[32;1m\u001b[1;3mLangChain is an open-source Python library that helps developers build applications with large language models (LLMs) through composability. Some key features of LangChain include:\n", | ||
"\n", | ||
"- Retrieval augmented generation - Allowing LLMs to retrieve and utilize external data sources when generating outputs.\n", | ||
"\n", | ||
"- Analyzing structured data - Tools for working with structured data like databases, APIs, PDFs, etc. and allowing LLMs to reason over this data.\n", | ||
"\n", | ||
"- Building chatbots and agents - Frameworks for building conversational AI applications.\n", | ||
"\n", | ||
"- Composability - LangChain allows you to chain together different LLM capabilities and data sources in a modular and reusable way.\n", | ||
"\n", | ||
"The library aims to make it easier to build real-world applications that leverage the power of large language models in a scalable and robust way. It provides abstractions and primitives for working with LLMs from different providers like OpenAI, Anthropic, Cohere, etc. LangChain is open-source and has an active community contributing new features and improvements.\u001b[0m\n", | ||
"\n", | ||
"\u001b[1m> Finished chain.\u001b[0m\n" | ||
] | ||
}, | ||
{ | ||
"data": { | ||
"text/plain": [ | ||
"{'input': 'what is LangChain?',\n", | ||
" 'output': 'LangChain is an open-source Python library that helps developers build applications with large language models (LLMs) through composability. Some key features of LangChain include:\\n\\n- Retrieval augmented generation - Allowing LLMs to retrieve and utilize external data sources when generating outputs.\\n\\n- Analyzing structured data - Tools for working with structured data like databases, APIs, PDFs, etc. and allowing LLMs to reason over this data.\\n\\n- Building chatbots and agents - Frameworks for building conversational AI applications.\\n\\n- Composability - LangChain allows you to chain together different LLM capabilities and data sources in a modular and reusable way.\\n\\nThe library aims to make it easier to build real-world applications that leverage the power of large language models in a scalable and robust way. It provides abstractions and primitives for working with LLMs from different providers like OpenAI, Anthropic, Cohere, etc. LangChain is open-source and has an active community contributing new features and improvements.'}" | ||
] | ||
}, | ||
"execution_count": 10, | ||
"metadata": {}, | ||
"output_type": "execute_result" | ||
} | ||
], | ||
"source": [ | ||
"# Create an agent executor by passing in the agent and tools\n", | ||
"agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)\n", | ||
"agent_executor.invoke({\"input\": \"what is LangChain?\"})" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"```{=mdx}\n", | ||
":::tip\n", | ||
"[LangSmith trace](https://smith.langchain.com/public/2f956a2e-0820-47c4-a798-c83f024e5ca1/r)\n", | ||
":::\n", | ||
"```\n", | ||
"\n", | ||
"## Using with chat history\n", | ||
"\n", | ||
"This type of agent can optionally take chat messages representing previous conversation turns. It can use that previous history to respond conversationally. For more details, see [this section of the agent quickstart](/docs/modules/agents/quick_start#adding-in-memory)." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 11, | ||
"metadata": {}, | ||
"outputs": [ | ||
{ | ||
"name": "stdout", | ||
"output_type": "stream", | ||
"text": [ | ||
"\n", | ||
"\n", | ||
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n" | ||
] | ||
}, | ||
{ | ||
"name": "stderr", | ||
"output_type": "stream", | ||
"text": [ | ||
"/Users/bagatur/langchain/libs/partners/anthropic/langchain_anthropic/chat_models.py:347: UserWarning: stream: Tool use is not yet supported in streaming mode.\n", | ||
" warnings.warn(\"stream: Tool use is not yet supported in streaming mode.\")\n" | ||
] | ||
}, | ||
{ | ||
"name": "stdout", | ||
"output_type": "stream", | ||
"text": [ | ||
"\u001b[32;1m\u001b[1;3mBased on what you told me, your name is Bob. I don't need to use any tools to look that up since you directly provided your name.\u001b[0m\n", | ||
"\n", | ||
"\u001b[1m> Finished chain.\u001b[0m\n" | ||
] | ||
}, | ||
{ | ||
"data": { | ||
"text/plain": [ | ||
"{'input': \"what's my name? Don't use tools to look this up unless you NEED to\",\n", | ||
" 'chat_history': [HumanMessage(content='hi! my name is bob'),\n", | ||
" AIMessage(content='Hello Bob! How can I assist you today?')],\n", | ||
" 'output': \"Based on what you told me, your name is Bob. I don't need to use any tools to look that up since you directly provided your name.\"}" | ||
] | ||
}, | ||
"execution_count": 11, | ||
"metadata": {}, | ||
"output_type": "execute_result" | ||
} | ||
], | ||
"source": [ | ||
"from langchain_core.messages import AIMessage, HumanMessage\n", | ||
"\n", | ||
"agent_executor.invoke(\n", | ||
" {\n", | ||
" \"input\": \"what's my name? Don't use tools to look this up unless you NEED to\",\n", | ||
" \"chat_history\": [\n", | ||
" HumanMessage(content=\"hi! my name is bob\"),\n", | ||
" AIMessage(content=\"Hello Bob! How can I assist you today?\"),\n", | ||
" ],\n", | ||
" }\n", | ||
")" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"```{=mdx}\n", | ||
":::tip\n", | ||
"[LangSmith trace](https://smith.langchain.com/public/e21ececb-2e60-49e5-9f06-a91b0fb11fb8/r)\n", | ||
":::\n", | ||
"```" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "poetry-venv-2", | ||
"language": "python", | ||
"name": "poetry-venv-2" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.9.1" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 4 | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left out Cohere because we don't handle ToolMessages yet