Default provider has CORS Problem in Firefox and Safari

[mdcoon]

When using default provider in Firefox or Safari to do something as simple as get a wallet balance, there are CORS errors due to unacceptable 'user-agent' header being passed from these browsers. Chrome does not pass this header so it works without issue.

I confirmed that if I use axios to manually construct the RPC call and override headers to only include Content-Type, the request goes through without issue. I see that it's possible for providers to override default options to ethersjs web connector. The fix might be as simple as supplying default "Content-Type" header if there are no other headers offered by the provider impl. Basically something needs to replace the browser default headers.

Using ethers version: 4.0.47
Firefox version: 76.0.1 for Mac
Safari version: 13.0.3

[ricmoo]

I use Safari almost exclusively, and I also just tried in Firefox on my local computer and both work fine.

It may be something else with your configuration. Can you provide more background?

[ricmoo]

e.g. in both Safari and Firefox:

const provider = ethers.getDefaultProvider();
provider.getBalance("ricmoo.firefly.eth").then((balance) => {
   console.log(balance.toString());
});
// "955864037352077165"

[mdcoon]

@ricmoo That exact code fails in both Firefox and Safari for me but succeeds without issue in Chrome. I don't know what is different. Browser version? ethers.js version? I provided all my versions above, do yours match? Thing is, our product was failing for our users when we used code like what you provided...so I had to work around with axios. So this isn't just my environment, it would be a handful of other users as well.

My current solution uses a custom provider that extends JsonRpcProvider and overrides its 'send' function to use axios. That works perfectly. I don't know what the difference is between what ethers.js is doing and what axios might be doing but there is clearly a difference somewhere.

[ricmoo]

Safari 13.1

> navigator.appVersion
< "5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1 Safari/605.1.15"

Firefox 72

navigator.userAgent
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:72.0) Gecko/20100101 Firefox/72.0"

Can you provider more context? Like a screen shot of the browser, with the errors in the console and the address bar? What network are you connecting to?

[mdcoon]

Not sure what other context you need...I literally used your example and I get errors. I think I've given plenty of info on what the issue is. If you monitor your network while calling your code snippet in Firefox/Safari, what Access-Control headers are being sent to Infura? If it includes user-agent, it should fail because infura doesn't allow that header for CORS requests. If it doesn't send that header, then somehow your browser is configured differently. If it does and still doesn't get CORS error, I'm at a loss on why Infura would give you a different response.

[ricmoo]

I mean, I need at least enough context in order to reproduce it. :)

Can you try this link? It works for me in both Safari (a compatible version to yours) and Firefox (an older version than yours).

Can you take a screen shot of the browser with the error console open and the browser bar visible?

Do you perhaps have any plug-ins that could be messing with things? Even if INFURA was not working, the code should work fine, since it would would hit Etherscan...

[ricmoo]

Any news? I may close this tomorrow if I can't reproduce it.

[zemse]

I had the exact same problem about an year ago but with web3.js ethereum/web3.js/2749. I understand that linking web3.js issues here might be irrelevant, but there were quite some issues on this same topic. nivida fixed issue ethereum/web3.js/2071 in PR#2564. Some more issues ethereum/web3.js/2978 which kindof match your title.

I'm not sure if my problem was solved because I moved to ethers that time and didn't face this issue after that.

[ricmoo]

I'm going to close this now, but if you can help reproduce it, please let me know and re-open...

[mverzilli]

We're experiencing the same. @ricmoo, your example works because getDefaultProvider() ends up using xhr, whereas the issue manifests when using JsonRpcProvider or a subclass. The issue should reproduce if you modify your snippet as follows:

<html>
  <body>
    <script src="https://static.ricmoo.com/uploads/ethers.min-03d01037c933.js"></script>
    <script type="application/javascript">
      const provider = new InfuraProvider('homestead',  your-infura-project-id);
      provider.getBlockNumber().then((n) => {
        console.log(n);
      });
    </script>
  </body>
</html>

[mdcoon]

I never responded above because your code snippet worked in the simplest case so I assumed it must have been my build environment. But since others chimed in, I know it's not just me. I'm using default provider but for a specific network (homestead for example) and it fails. Does that help? I'm using within a create-react-app (CRA) so not sure if the build process for CRA conflicts with something to create this weird scenario.

[ricmoo]

@mverzilli

There is no xhr used in ethers, so it sounds like your bundler is pulling in some shim for fetch?

Actually, looking at your code, you are targeting an ancient version of ethers that was uploaded as a one-time demo (where did you get that link?). Can you try using the version linked to in the readme?

[mverzilli]

I got it from here #858 (comment). I opened your link, took the source code and modified it to cause the error condition. I figured it'd be helpful if I based my example on the same test code you provided. Sorry if that was a bad call!

The version we're using in the context where this issue arose is ethers@^4.0.47.

Now, I think your question about this being caused by bundlers might be pointing in the right direction. Here's why I think that may be the case:

Guided by @mdcoon's hint about overriding JsonRpcProvider, I extended JsonRpcProvider, overriding the send method with exactly the same code that's used in the library (so no Axios, no ad-hoc header manipulation, no "patches"). And that worked, which was a bit surprising. So it certainly looks like Webpack might be shimming requests issued from dependency code.

It certainly doesn't look like the problem is caused by ethers, but it might be worth adding a troubleshooting note in the docs somewhere, especially if this happens with CRA as @mdcoon reports. I'll try to narrow the issue down as soon as I get more time to work on this.

[funky86]

@mverzilli would you please share the code of your extended JsonRpcProvider? That would be so much appreciated.

[mverzilli]

Sure, keep in mind it's just a copy of the original code, just vendored into your project to remove any bundling weirdness. It's probably not even the right solution, it's more of a hack. See the code below, code is provided as is with no warranties whatsoever :).

import { JsonRpcProvider } from 'ethers/providers';
import { ConnectionInfo } from 'ethers/utils/web';
import * as errors from 'ethers/errors';
import { encode as base64Encode } from 'ethers/utils/base64';
import { toUtf8Bytes } from 'ethers/utils/utf8';

// Some environments (Trust Wallet and company) use a global map
// to track JSON-RPC ID, so we try to keep IDs unique across all
// connections. See #489.
let _nextId = 42;

type Header = { key: string; value: string };

export default class CustomJsonRpcProvider extends JsonRpcProvider {
  getResult(payload: { error?: { code?: number; data?: any; message?: string }; result?: any }): any {
    if (payload.error) {
      // @TODO: not any
      const error: any = new Error(payload.error.message);
      error.code = payload.error.code;
      error.data = payload.error.data;
      throw error;
    }

    return payload.result;
  }

  async fetchJson(connection: string | ConnectionInfo, json: string, processFunc: (value: any) => any): Promise<any> {
    const headers: { [key: string]: Header } = {};

    let url = '';

    let timeout = 2 * 60 * 1000;

    if (typeof connection === 'string') {
      url = connection;
    } else if (typeof connection === 'object') {
      if (connection.url == null) {
        errors.throwError('missing URL', errors.MISSING_ARGUMENT, { arg: 'url' });
      }

      url = connection.url;

      if (typeof connection.timeout === 'number' && connection.timeout > 0) {
        timeout = connection.timeout;
      }

      if (connection.headers) {
        for (const key in connection.headers) {
          headers[key.toLowerCase()] = { key: key, value: String(connection.headers[key]) };
        }
      }

      if (connection.user != null && connection.password != null) {
        if (url.substring(0, 6) !== 'https:' && connection.allowInsecure !== true) {
          errors.throwError('basic authentication requires a secure https url', errors.INVALID_ARGUMENT, {
            arg: 'url',
            url: url,
            user: connection.user,
            password: '[REDACTED]',
          });
        }

        const authorization = connection.user + ':' + connection.password;
        headers['authorization'] = {
          key: 'Authorization',
          value: 'Basic ' + base64Encode(toUtf8Bytes(authorization)),
        };
      }
    }

    return new Promise(function (resolve, reject) {
      const request = new XMLHttpRequest();

      let timer: any = null;
      timer = setTimeout(() => {
        if (timer == null) {
          return;
        }
        timer = null;

        reject(new Error('timeout'));
        setTimeout(() => {
          request.abort();
        }, 0);
      }, timeout);

      const cancelTimeout = () => {
        if (timer == null) {
          return;
        }
        clearTimeout(timer);
        timer = null;
      };

      if (json) {
        request.open('POST', url, true);
        headers['content-type'] = { key: 'Content-Type', value: 'application/json' };
      } else {
        request.open('GET', url, true);
      }

      Object.keys(headers).forEach((key) => {
        const header = headers[key];
        request.setRequestHeader(header.key, header.value);
      });

      request.onreadystatechange = function () {
        if (request.readyState !== 4) {
          return;
        }

        if (request.status != 200) {
          cancelTimeout();
          // @TODO: not any!
          const error: any = new Error('invalid response - ' + request.status);
          error.statusCode = request.status;
          if (request.responseText) {
            error.responseText = request.responseText;
          }
          reject(error);
          return;
        }

        let result: any = null;
        try {
          result = JSON.parse(request.responseText);
        } catch (error) {
          cancelTimeout();
          // @TODO: not any!
          const jsonError: any = new Error('invalid json response');
          jsonError.orginialError = error;
          jsonError.responseText = request.responseText;
          if (json != null) {
            jsonError.requestBody = json;
          }
          jsonError.url = url;
          reject(jsonError);
          return;
        }

        if (processFunc) {
          try {
            result = processFunc(result);
          } catch (error) {
            cancelTimeout();
            error.url = url;
            error.body = json;
            error.responseText = request.responseText;
            reject(error);
            return;
          }
        }

        cancelTimeout();
        resolve(result);
      };

      request.onerror = function (error) {
        cancelTimeout();
        reject(error);
      };

      try {
        if (json != null) {
          request.send(json);
        } else {
          request.send();
        }
      } catch (error) {
        cancelTimeout();
        // @TODO: not any!
        const connectionError: any = new Error('connection error');
        connectionError.error = error;
        reject(connectionError);
      }
    });
  }

  send(method: string, params: any): Promise<any> {
    const request = {
      method: method,
      params: params,
      id: _nextId++,
      jsonrpc: '2.0',
    };

    return this.fetchJson(this.connection, JSON.stringify(request), this.getResult).then((result) => {
      this.emit('debug', {
        action: 'send',
        request: request,
        response: result,
        provider: this,
      });
      return result;
    });
  }
}

[funky86]

@mverzilli thanks a lot :) ! Understandable.
I have a problem - I get an error. Do you maybe know what am I doing wrong?
I have the code of the Firebase function in JavaScript so I had to remove the type definitions of arguments in your code since it's TypeScript. I'm attaching the whole script. If I use ethers.providers.JsonRpcProvider in line 300, everything works, Smart Contract method is properly called, just CORS error happen if I deploy this function to the Firebase backend and call it from the website.
If I run the code with CustomJsonRpcProvider in line 300, I get the following error:

`

Error: could not detect network (event="noNetwork", code=NETWORK_ERROR, version=providers/5.1.0)
at Logger.makeError (/Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/node_modules/@ethersproject/logger/lib/index.js:180:21)
at Logger.throwError (/Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/node_modules/@ethersproject/logger/lib/index.js:189:20)
at CustomJsonRpcProvider. (/Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/node_modules/@ethersproject/providers/lib/json-rpc-provider.js:412:54)
at step (/Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/node_modules/@ethersproject/providers/lib/json-rpc-provider.js:48:23)
at Object.next (/Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/node_modules/@ethersproject/providers/lib/json-rpc-provider.js:29:53)
at fulfilled (/Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/node_modules/@ethersproject/providers/lib/json-rpc-provider.js:20:58) {
reason: 'could not detect network',
code: 'NETWORK_ERROR',
event: 'noNetwork'
}
(node:25631) UnhandledPromiseRejectionWarning: TypeError: Cannot read property '0' of undefined
at /Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/index.js:229:43
`

index.js.zip

[mverzilli]

Not sure if that's the issue, but keep in mind I was using this on Ethers 4, not Ethers 5, which based on your stack trace seems to be the version you're on

[funky86]

Correct, I was using v5. I tried with ethers 4.0.48 and I got the error down below 😔

ReferenceError: _nextId is not defined
at CustomJsonRpcProvider.send (/Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/index.js:185:20)
at Timeout._onTimeout (/Users/localuser/Documents/Privat/Projects/1000blocks/backend/functions/node_modules/ethers/providers/json-rpc-provider.js:207:27)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7)

[funky86]

Fixed! The problem was that the client sent over a HTTP POST parameter value "undefined" instead of the address of user's crypto wallet. Our custom Smart Contract method got this value as method parameter. For some reason the CORS error happened when we called this method with value "undefined".