AzureOpenAI azure_ad_token_provider Keyerror

[axen22]

System Info

When I use below snippet of code

import os
from azure.identity import DefaultAzureCredential
from azure.identity import get_bearer_token_provider
from langchain.llms import AzureOpenAI
from langchain.chat_models import AzureChatOpenAI

credential = DefaultAzureCredential(interactive_browser_tenant_id=tenant_id, 
                                                              interactive_browser_client_id=client_id, 
                                                              client_secret=client_secret)

token_provider = get_bearer_token_provider(credential, "https://cognitiveservices.azure.com/.default")

endpoint =  "https://xxxx.openai.azure.com"
client = AzureOpenAI(  azure_endpoint=endpoint,
                                       api_version="2023-05-15",
                                       azure_deployment="example-gpt-4",
                                       azure_ad_token_provider=token_provider)

I get error :

KeyError                                  Traceback (most recent call last)
Cell In[36], line 21
     18 # api_version = "2023-05-15"
     19 endpoint =  "https://xxxx.openai.azure.com"
---> 21 client = AzureOpenAI(
     22     azure_endpoint=endpoint,
     23     api_version="2023-05-15",
     24     azure_deployment="example-gpt-4",
     25     azure_ad_token_provider=token_provider,
     26 )

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/langchain_core/load/serializable.py:97, in Serializable.__init__(self, **kwargs)
     96 def __init__(self, **kwargs: Any) -> None:
---> 97     super().__init__(**kwargs)
     98     self._lc_kwargs = kwargs

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/pydantic/v1/main.py:339, in BaseModel.__init__(__pydantic_self__, **data)
    333 """
    334 Create a new model by parsing and validating input data from keyword arguments.
    335 
    336 Raises ValidationError if the input data cannot be parsed to form a valid model.
    337 """
    338 # Uses something other than `self` the first arg to allow "self" as a settable attribute
--> 339 values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    340 if validation_error:
    341     raise validation_error

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/pydantic/v1/main.py:1102, in validate_model(model, input_data, cls)
   1100     continue
   1101 try:
-> 1102     values = validator(cls_, values)
   1103 except (ValueError, TypeError, AssertionError) as exc:
   1104     errors.append(ErrorWrapper(exc, loc=ROOT_KEY))

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/langchain/llms/openai.py:887, in AzureOpenAI.validate_environment(cls, values)
    877             values["openai_api_base"] += (
    878                 "/deployments/" + values["deployment_name"]
    879             )
    880         values["deployment_name"] = None
    881 client_params = {
    882     "api_version": values["openai_api_version"],
    883     "azure_endpoint": values["azure_endpoint"],
    884     "azure_deployment": values["deployment_name"],
    885     "api_key": values["openai_api_key"],
    886     "azure_ad_token": values["azure_ad_token"],
--> 887     "azure_ad_token_provider": values["azure_ad_token_provider"],
    888     "organization": values["openai_organization"],
    889     "base_url": values["openai_api_base"],
    890     "timeout": values["request_timeout"],
    891     "max_retries": values["max_retries"],
    892     "default_headers": values["default_headers"],
    893     "default_query": values["default_query"],
    894     "http_client": values["http_client"],
    895 }
    896 values["client"] = openai.AzureOpenAI(**client_params).completions
    897 values["async_client"] = openai.AsyncAzureOpenAI(
    898     **client_params
    899 ).completions

KeyError: 'azure_ad_token_provider'

Ive also tried AzureChatOpenAI , and I get the same error back.
The error is not reproduced when I use openai library AzureOpenAI .

Also on openai the azure_ad_token_provider has type azure_ad_token_provider: 'AzureADTokenProvider | None' = None while in langchain it has type azure_ad_token_provider: Optional[str] = None which also makes me wonder if it would take as input a different type than string to work with.

any ideas on how to fix this? Im actually using Azure Service principal authentication, and if I use as alternative field azure_ad_token = credential.get_token(“https://cognitiveservices.azure.com/.default”).token I get token expired after 60min which does not happen with a bearer token, so It is important to me to make the token_provider work.

libraries :
pydantic 1.10.12
pydantic_core 2.10.1
openai 1.2.0
langchain 0.0.342
langchain-core 0.0.7

Who can help?

@hwchase17 @agola11

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

import os
from azure.identity import DefaultAzureCredential
from azure.identity import get_bearer_token_provider
from langchain.llms import AzureOpenAI
from langchain.chat_models import AzureChatOpenAI

credential = DefaultAzureCredential(interactive_browser_tenant_id=tenant_id,
interactive_browser_client_id=client_id,
client_secret=client_secret)

token_provider = get_bearer_token_provider(credential, "https://cognitiveservices.azure.com/.default")

endpoint = "https://xxxx.openai.azure.com"
client = AzureOpenAI( azure_endpoint=endpoint,
api_version="2023-05-15",
azure_deployment="example-gpt-4",
azure_ad_token_provider=token_provider)

Expected behavior

client = AzureOpenAI( azure_endpoint=endpoint,
api_version="2023-05-15",
azure_deployment="example-gpt-4",
azure_ad_token_provider=token_provider)

should return a Runnable instance which I can use for LLMChain

[dosubot]

🤖

Hello @axen22! 👋 I'm Dosu, an AI here to assist you with your issue while we wait for a human maintainer. This is my first time interacting with you, and I'm here to help you solve bugs, answer questions, and guide you through the contribution process. Let's get started!

Based on the information you've provided and the context from the LangChain repository, it seems like the azure_ad_token_provider parameter in the AzureOpenAI and AzureChatOpenAI classes is expected to be a function that returns an Azure Active Directory token. This function will be invoked on every request to avoid token expiration.

Here is an example of how you can do this:

def my_token_provider():
    # Your logic to get a new token from Azure Active Directory
    return new_token

azure_chat = AzureChatOpenAI(
    azure_deployment="35-turbo-dev",
    openai_api_version="2023-05-15",
    azure_ad_token_provider=my_token_provider,
)

In the above code, replace my_token_provider with your function that fetches a new token from Azure Active Directory. This function will be used to refresh the token on every request, thus avoiding token expiration.

However, there seems to be a discrepancy in the type hint for azure_ad_token_provider in the LangChain library. The type hint suggests it should be a string or None, which might be a mistake in the documentation or code. The function itself would likely be passed as a string name of the function, or a direct function object (which would require a different type hint).

I also found a similar solved issue in the LangChain repository: Azure OpenAI token authenticate issue. which suggests setting the OPENAI_API_KEY and OPENAI_API_VERSION as environment variables and using the AzureOpenAI class with the "azure_ad" api_type.

Please try the above suggestions and let me know if it helps. If the issue persists, it might be a good idea to raise this as a new issue in the LangChain repository, as it could be a bug or a documentation error.

Sources

• Azure OpenAI token authenticate issue.
• libs/langchain/langchain/chat_models/azure_openai.py
• libs/langchain/langchain/embeddings/azure_openai.py

[clemlesne]

Same behaviour from my side. I confirm the issue.

[169]

@axen22 @clemlesne I've submitted a PR to address this issue.

If you are in a hurry, you can use it like this:

client = AzureOpenAI(azure_endpoint=endpoint,
                                        api_version="2023-05-15",
                                        azure_deployment="example-gpt-4",
                                        azure_ad_token_provider=token_provider())

[axen22]

@169 fix will be part of version "0.0.344" ? tested "0.0.344" now, still same issue appears.

also tried azure_ad_token_provider=token_provider() but when I feed client to llmchain with below snippet:

from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate

prompt_template = "What is a good name for a company that makes {product}?"

llm_chain = LLMChain(llm=client, prompt=PromptTemplate.from_template(prompt_template))
llm_chain("colorful socks")

I get:

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/openai/lib/azure.py:237, in AzureOpenAI._prepare_options(self, options)
    234 headers: dict[str, str | Omit] = {**options.headers} if is_given(options.headers) else {}
    235 options.headers = headers
--> 237 azure_ad_token = self._get_azure_ad_token()
    238 if azure_ad_token is not None:
    239     if headers.get("Authorization") is None:

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/openai/lib/azure.py:223, in AzureOpenAI._get_azure_ad_token(self)
    221 provider = self._azure_ad_token_provider
    222 if provider is not None:
--> 223     token = provider()
    224     if not token or not isinstance(token, str):  # pyright: ignore[reportUnnecessaryIsInstance]
    225         raise ValueError(
    226             f"Expected `azure_ad_token_provider` argument to return a string but it returned {token}",
    227         )

TypeError: 'str' object is not callable

[169]

@axen22 This PR has not been merged yet. I will mention you after it is merged and published.

Sorry, I know, token_provider() doesn’t work either. There is no way to solve it now. We need to wait for langchain to release a new version.

[169]

@axen22 @clemlesne The new version has been released, please upgrade to the latest version pip install langchain==v0.0.346