JwtTokenExtractor.getIdentity:err! FetchError: request to https://login.botframework.com/v1/.well-known/openidconfiguration failed, reason: connect ETIMEDOUT

[saulo2]

Hi,

I am getting the error below with the botbuilder 4.15.0:

JwtTokenExtractor.getIdentity:err! FetchError: request to https://login.botframework.com/v1/.well-known/openidconfiguration failed, reason: connect ETIMEDOUT 104.41.13.179:443
at ClientRequest. (/opt/nexos-teams/bot/node_modules/node-fetch/lib/index.js:1501:11)
at ClientRequest.emit (node:events:513:28)
at TLSSocket.socketErrorListener (node:_http_client:494:9)
at TLSSocket.emit (node:events:513:28)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
type: 'system',
errno: 'ETIMEDOUT',
code: 'ETIMEDOUT'
}

I am behind a corporate fiewall. I have tried setting the HTTP_PROXY and HTTPS_PROXY variables, but it did not solve the problem.

I have also tries configuring the proxySettings options as below, but it also did not solve the problem.

const botFrameworkAuthentication = createBotFrameworkAuthenticationFromConfiguration(null, credentialsFactory, null, null, {
proxySettings: {
host: "cachesv",
port: 8080
}
});

Does botbuilder supports http proxies?

[saulo2]

After some investigation, the problem seems to be in the refreshCache method of the OpenIdMetadata class. More specifically, in the line:

const res = await fetch(this.url);

which does not use the proxySettings provided for the createBotFrameworkAuthenticationFromConfiguration method.

[tracyboehrer]

@saulo2 There is a test version on NPM we think addresses this. The version is: 4.21.4-dev. Can you try this version and check if it corrects the problem?

[ceciliaavila]

This is how we set the proxy settings for testing the fix:

[saulo2]

@saulo2 There is a test version on NPM we think addresses this. The version is: 4.21.4-dev. Can you try this version and check if it corrects the problem?

@tracyboehrer and @ceciliaavila thanks for addressing this issue. Right now, I am travelling on vacation. As soon I come back (circa 10 days) I am going to test the fix and give you feedback.

[denisogolubev]

@tracyboehrer @ceciliaavila hi)
We're facing the same issue.

To solve it we've followed your instructions, installed:

{
    "botbuilder": "4.21.4-dev",
}

Also, we have added proxySettings to the botFrameworkAuthentication.
And now the error a bit changed:

FetchError: request to https://login.botframework.com/v1/.well-known/keys failed, reason: connect ECONNREFUSED XX.XXX.XXX.XX:XXX
    at ClientRequest.<anonymous> (/node_modules/node-fetch/lib/index.js:1501:11)
    at ClientRequest.emit (node:events:517:28)
    at ClientRequest.emit (node:domain:489:12)
    at TLSSocket.socketErrorListener (node:_http_client:501:9)
    at TLSSocket.emit (node:events:517:28)
    at TLSSocket.emit (node:domain:489:12)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED'
}

It started to fail on request to https://login.botframework.com/v1/.well-known/keys.
Could you help us?

[tracyboehrer]

@saulo2 and @denisogolubev There is a new test version: 4.21.4-dev2

[denisogolubev]

@tracyboehrer thanks a lot, it fixed the previous issue.
Now we see this error:

[onTurnError] unhandled error: ClientAuthError: endpoints_resolution_error: Error: could not resolve endpoints. Please check network and try again. Detail: ClientConfigurationError: untrusted_authority: The provided authority is not a trusted authority. Please include this authority in the knownAuthorities config parameter.
ClientAuthError: endpoints_resolution_error: Error: could not resolve endpoints. Please check network and try again. Detail: ClientConfigurationError: untrusted_authority: The provided authority is not a trusted authority. Please include this authority in the knownAuthorities config parameter.
    at ClientAuthError.AuthError [as constructor] (/node_modules/@azure/msal-common/src/error/AuthError.ts:49:9)
    at new ClientAuthError (/node_modules/@azure/msal-common/src/error/ClientAuthError.ts:223:15)
    at Function.ClientAuthError.createEndpointDiscoveryIncompleteError (/node_modules/@azure/msal-common/src/error/ClientAuthError.ts:269:16)
    at Function.<anonymous> (/node_modules/@azure/msal-common/src/authority/AuthorityFactory.ts:58:35)
    at step (/node_modules/@azure/msal-common/dist/index.cjs.js:79:23)
    at Object.throw (/node_modules/@azure/msal-common/dist/index.cjs.js:60:53)
    at rejected (/node_modules/@azure/msal-common/dist/index.cjs.js:51:65)
    at processTicksAndRejections (node:internal/process/task_queues:95:5) {
  errorCode: 'endpoints_resolution_error',
  errorMessage: 'Error: could not resolve endpoints. Please check network and try again. Detail: ClientConfigurationError: untrusted_authority: The provided authority is not a trusted authority. Please include this authority in the knownAuthorities config parameter.',
  subError: '',
  correlationId: 'fddc35e1-2192-4ff5-a8f8-0c196ad8da6d'
}

Is it something with our configurations or is it still the issue with the proxy?
Because I don't see knownAuthorities as a property in any existing configs we use.

[ceciliaavila]

Hi @denisogolubev,
This same error was reported by another customer that was using a proxy.
The problem seems to be in a call that the @azure/msal-node library does internally.
Can you try the following?

• Check if you can access this URL in a browser or your proxy is blocking it:
  https://login.microsoftonline.com/common/discovery/instance?api-version=1.1&authorization_endpoint=https://login.microsoftonline.com/botframework.com/oauth2/v2.0/authorize

• There's a proxyUrl property that can be configured in the bot (more info). Here is an example of how to provide this setting:

[ceciliaavila]

According to this issue, proxy settings are not globally set in NodeJS, we need to use libraries to manage this.
Using the node-global-proxy package, we can set the HTTP_PROXY and HTTPS_PROXY variables for Node to use.
There's no need to make changes in the SDK, just a simple configuration in the bot's index.js file.

This is a better alternative than setting the proxy host and port values in the BotframeworkAuthentication options. However, for the untrusted_authority error, you should try the workaround proposed by msal-node.

[denisogolubev]

@ceciliaavila it looks like this configuration with node-global-proxy solves the issue)
Thanks a lot)

[saulo2]

@tracyboehrer and @ceciliaavila I confirm version 4.21.4-dev2 fixes the matter. However, I have tested 4.21.4 and it seems it does not contain the fix. Also, it seems the fix is not in the main branch. For example, the constructor of the class OpenIdMetadata (https://github.com/microsoft/botbuilder-js/blob/main/libraries/botframework-connector/src/auth/openIdMetadata.ts) does not receive a ProxySettings instance as it second argument.

[ceciliaavila]

Hi @saulo2, we discarded the fix from 4.21.4-dev2 because we found a simpler solution by adding to the bot the node-global-proxy package. Can you try this solution to see if it fixes the issue on your side?

[saulo2]

@ceciliaavila the solution around node-global-proxy also fixed the issue, but it caused undesirable side effects. My bot access APIs on other intranet servers, which the corporate proxy does not have access to because it only access servers on the internet. Thus, with node-global-proxy my bot is able to access https://login.botframework.com/v1/.well-known/openidconfiguration but it does not work, because it is not able to access the other intranet servers. Could you reintroduce your fix in a new botbuilder-js version?

[ceciliaavila]

@saulo2, thanks for your input.
@tracyboehrer, we re-opened the PR with the fix, if you think it's correct, you can merge it. Thanks!

[tracyboehrer]

@ceciliaavila @saulo2 @denisogolubev Can we clarify what does work? denisogolubev reported that 4.21.4-dev2 (which was built from the PR branch) caused a subsequent problem, but that node-global-proxy worked. Is the final solution this PR and possibly node-global-proxy?

BTW, the actual 4.21.4 release done recently does NOT have that PR fix.

[saulo2]

@tracyboehrer I am happy to clarify that:

• 4.21.4-dev2 fully works for me, without any further issues (I have not faced the last problem @denisogolubev reported);

• node-global-proxy partially works for me, i.e. botbuilder-js is able to access https://login.botframework.com/v1/.well-known/openidconfiguration, but it forces every http access of my bot to go trought the proxy, causing further issues, e.g. my bot is not able to access other intranet servers, thus it is a no go for me.

[saulo2]

@tracyboehrer and @ceciliaavila I am happy to confirm that the fix has been successfully incorporated into the codebase. I tested version 4.22.1 and it is working as expected.