forked from langchain-ai/langchain
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
community[minor] : adds callback handler for Fiddler AI (langchain-ai…
…#17708) **Description:** Callback handler to integrate fiddler with langchain. This PR adds the following - 1. `FiddlerCallbackHandler` implementation into langchain/community 2. Example notebook `fiddler.ipynb` for usage documentation [Internal Tracker : FDL-14305] **Issue:** NA **Dependencies:** - Installation of langchain-community is unaffected. - Usage of FiddlerCallbackHandler requires installation of latest fiddler-client (2.5+) **Twitter handle:** @fiddlerlabs @behalder Co-authored-by: Barun Halder <barun@fiddler.ai>
- Loading branch information
Showing
4 changed files
with
496 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,215 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"id": "0cebf93b", | ||
"metadata": {}, | ||
"source": [ | ||
"## Fiddler Langchain integration Quick Start Guide\n", | ||
"\n", | ||
"Fiddler is the pioneer in enterprise Generative and Predictive system ops, offering a unified platform that enables Data Science, MLOps, Risk, Compliance, Analytics, and other LOB teams to monitor, explain, analyze, and improve ML deployments at enterprise scale. " | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "38d746c2", | ||
"metadata": {}, | ||
"source": [ | ||
"## 1. Installation and Setup" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "e0151955", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"# langchain>=0.1.7 langchain-openai fiddler-client" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "5662f2e5-d510-4eef-b44b-fa929e5b4ad4", | ||
"metadata": {}, | ||
"source": [ | ||
"## 2. Fiddler connection details " | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "64fac323", | ||
"metadata": {}, | ||
"source": [ | ||
"*Before you can add information about your model with Fiddler*\n", | ||
"\n", | ||
"1. The URL you're using to connect to Fiddler\n", | ||
"2. Your organization ID\n", | ||
"3. Your authorization token\n", | ||
"\n", | ||
"These can be found by navigating to the *Settings* page of your Fiddler environment." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "f6f8b73e-d350-40f0-b7a4-fb1e68a65a22", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"URL = \"\" # Your Fiddler instance URL, Make sure to include the full URL (including https://). For example: https://demo.fiddler.ai\n", | ||
"ORG_NAME = \"\"\n", | ||
"AUTH_TOKEN = \"\" # Your Fiddler instance auth token\n", | ||
"\n", | ||
"# Fiddler project and model names, used for model registration\n", | ||
"PROJECT_NAME = \"\"\n", | ||
"MODEL_NAME = \"\" # Model name in Fiddler" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "0645805a", | ||
"metadata": {}, | ||
"source": [ | ||
"## 3. Create a fiddler callback handler instance" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "13de4f9a", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"from langchain.callbacks.fiddler_callback import FiddlerCallback\n", | ||
"\n", | ||
"fiddler_handler = FiddlerCallback(\n", | ||
" url=URL,\n", | ||
" org_name=ORG_NAME,\n", | ||
" project_name=PROJECT_NAME,\n", | ||
" model_name=MODEL_NAME,\n", | ||
" auth_token=AUTH_TOKEN,\n", | ||
")" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "2276368e-f1dc-46be-afe3-18796e7a66f2", | ||
"metadata": {}, | ||
"source": [ | ||
"## Example 1 : Basic Chain" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "c9de0fd1", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"from langchain_core.output_parsers import StrOutputParser\n", | ||
"from langchain_openai import OpenAI\n", | ||
"\n", | ||
"# Note : Make sure openai API key is set in the environment variable OPENAI_API_KEY\n", | ||
"llm = OpenAI(temperature=0, streaming=True, callbacks=[fiddler_handler])\n", | ||
"output_parser = StrOutputParser()\n", | ||
"\n", | ||
"chain = llm | output_parser\n", | ||
"\n", | ||
"# Invoke the chain. Invocation will be logged to Fiddler, and metrics automatically generated\n", | ||
"chain.invoke(\"How far is moon from earth?\")" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "309bde0b-e1ce-446c-98ac-3690c26a2676", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"# Few more invocations\n", | ||
"chain.invoke(\"What is the temperature on Mars?\")\n", | ||
"chain.invoke(\"How much is 2 + 200000?\")\n", | ||
"chain.invoke(\"Which movie won the oscars this year?\")\n", | ||
"chain.invoke(\"Can you write me a poem about insomnia?\")\n", | ||
"chain.invoke(\"How are you doing today?\")\n", | ||
"chain.invoke(\"What is the meaning of life?\")" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "48fa4782-c867-4510-9430-4ffa3de3b5eb", | ||
"metadata": {}, | ||
"source": [ | ||
"## Example 2 : Chain with prompt templates" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "2aa2c220-8946-4844-8d3c-8f69d744d13f", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"from langchain.prompts import (\n", | ||
" ChatPromptTemplate,\n", | ||
" FewShotChatMessagePromptTemplate,\n", | ||
")\n", | ||
"\n", | ||
"examples = [\n", | ||
" {\"input\": \"2+2\", \"output\": \"4\"},\n", | ||
" {\"input\": \"2+3\", \"output\": \"5\"},\n", | ||
"]\n", | ||
"\n", | ||
"example_prompt = ChatPromptTemplate.from_messages(\n", | ||
" [\n", | ||
" (\"human\", \"{input}\"),\n", | ||
" (\"ai\", \"{output}\"),\n", | ||
" ]\n", | ||
")\n", | ||
"\n", | ||
"few_shot_prompt = FewShotChatMessagePromptTemplate(\n", | ||
" example_prompt=example_prompt,\n", | ||
" examples=examples,\n", | ||
")\n", | ||
"\n", | ||
"final_prompt = ChatPromptTemplate.from_messages(\n", | ||
" [\n", | ||
" (\"system\", \"You are a wondrous wizard of math.\"),\n", | ||
" few_shot_prompt,\n", | ||
" (\"human\", \"{input}\"),\n", | ||
" ]\n", | ||
")\n", | ||
"\n", | ||
"# Note : Make sure openai API key is set in the environment variable OPENAI_API_KEY\n", | ||
"llm = OpenAI(temperature=0, streaming=True, callbacks=[fiddler_handler])\n", | ||
"\n", | ||
"chain = final_prompt | llm\n", | ||
"\n", | ||
"# Invoke the chain. Invocation will be logged to Fiddler, and metrics automatically generated\n", | ||
"chain.invoke({\"input\": \"What's the square of a triangle?\"})" | ||
] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "Python 3 (ipykernel)", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.9.6" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 5 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.