[Bug]: puppeteer 20.6.0 introduced a regression (with iframe?) that now is Error: Requesting main frame too early!

[andreabisello]

Minimal, reproducible example

i can't give the code to reproduce because is a business code, so probably this will be a "closed" ticket, but anyway i can describe it, i hope can be useful.

the same error is repored here by other users, no one could understand how to reproduce systematically.

Error string

Error: Requesting main frame too early!

Bug behavior

• Flaky
• PDF

Background

i open a browser, i open a page, a navigate to a webpage, i click a button, that opens a iframe, i take a reference to the iframe, i write some text, i click a button that close the iframe, and the error occur and the browser is closed (or the browser is closed and then the error occur).

that test worked for 2 years until the introduction of puppeteer 20.6.0

the website still the same

Expectation

when the website close the iframe, the execution of the code should keep working.

Reality

when, clicking a button, the website close the iframe, the browser close and the error is raised (or the error is raised and so the browser close)

until puppeteer 20.5.0 this was working.
with puppeteer 20.6.0 you obtain

failed: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed..

with puppeteer 21.3 or also 21.4.1 you obtain

Error: Requesting main frame too early!.

Puppeteer configuration file (if used)

No response

Puppeteer version

21.4.1

Node version

18.16.1

Package manager

npm

Package manager version

9.5.1

Operating system

Windows

[github-actions]

This issue was not reproducible. Please check that your example runs locally and the following:

• Ensure the script does not rely on dependencies outside of puppeteer and puppeteer-core.
• Ensure the error string is just the error message.

  • Bad:

    Error: something went wrong
      at Object.<anonymous> (/Users/username/repository/script.js:2:1)
      at Module._compile (node:internal/modules/cjs/loader:1159:14)
      at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
      at Module.load (node:internal/modules/cjs/loader:1037:32)
      at Module._load (node:internal/modules/cjs/loader:878:12)
      at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
      at node:internal/main/run_main_module:23:47

  • Good: Error: something went wrong.

• Ensure your configuration file (if applicable) is valid.
• If the issue is flaky (does not reproduce all the time), make sure 'Flaky' is checked.
• If the issue is not expected to error, make sure to write 'no error'.

Once the above checks are satisfied, please edit your issue with the changes and we will
try to reproduce the bug again.

---

Analyzer run

[OrKoN]

@andreabisello can you create a test page that resembles the real issue?

[OrKoN]

The error indicates that there is an action on the main frame but Puppeteer has not received the information about the main frame from the browser yet. Perhaps it is a race condition but it would also depend on how exactly the frame is accessed.

[andreabisello]

@OrKoN i cant because is company code that test company software. maybe i should create a website with an iframe and a button that open that iframe, makes something, and close the iframe.

they way i access the frame is that one

        let creationFrame
        while(!creationFrame){
            await page.waitForTimeout(1000)
            for(const frame of page.mainFrame().childFrames()){
                if(frame.url().includes("EditorFatturaPA")){
                    creationFrame = frame
                }
            }
        }

then i make every interation using "creationFrame" , for clicking typing etc ...
then when i click the button that close the frame

        await creationFrame.evaluate(
            () => {
                document.querySelector("body > div.bootbox.modal.fade.in > div.modal-dialog > div > div.modal-footer > button.btn.btn-sm.btn-success").click();
            }
        )

the problem raise.

maybe i could wrap that evaluate in a try - catch expecting some kind of exception. now i try.

[fwebdev]

I had a simmilar Error with Puppeteer Version 21.4.1 but that is fixed for me with Version 21.5.0 now

[bogdanionitabp]

This issue still persists in version 21.5.2 and it cannot be reproduced reliably.
It happens during the call to Page.goto() at random times, especially after the application has been running for a while (4-5 hours seems to be the sweet spot).
The main problem is that once this happens, all future calls to Page.goto() will immediately fail with the same error and there's no apparent way to work around this without restarting the application.

[benlhachemi]

I'm using 21.4.1
It happens to me too today several times when calling Page.goto()
I even tried a sleep promise for 30 second to make sure that the page is loaded but still sometimes there is the same error.

also another important thing: The problem only happens after launching a new browser instance, first time launch works fine with no problem at all, when my first task is finished I close the browser with browser.close() and when I start the next task I open the browser again with puppeteer.launch() + browser.newPage() => and here where the problem happens

Note: I'm using "puppeteer-extra" which is built in top of "Puppeteer"

[benlhachemi]

I found a solution!

1 - Upgrade to 21.5.2
2 - I'm using nestJs, and I have 2 modules (channel service and upload service):
channel service

//properties
browser: Browser
page: Page

// methods
public async initBrowser() {...}

public async connectToChannel(externalPage?: Page){
   const page = this.page || externalPage // that line causes the problem 
   this.page = page
   page.goto('studio.youtube.com') // here where the error is thrown
   ....
}

...other methods

upload service

// properties
private browser: Browser;
private page: Page;
constructor(private readonly channelService: ChannelService) {}

public async uploadVideo() {
   this.browser = await browser.launch({headless: false}) // changing launching params didn't do anything to solve the problem
   await this.channelService.connectToChannel(this.page)
   ... rest of code
}

what I basically did is I changed this line of code on my channelService and it works
const page = externalPage || this.page

[muthusk]

I was able to reproduce this when one of the args for puppeteer launch had "--single-process", when removing it it worked fine.

[fernandomondo]

This issue still persists in version in 21.5.2.
I dont use "--single-process".

Error: Requesting main frame too early!
    at assert (C:\Fernando\Projects\whatsapp-test\node_modules\puppeteer-core\src\util\assert.ts:29:11)
    at FrameManager.mainFrame (C:\Fernando\Projects\whatsapp-test\node_modules\puppeteer-core\src\cdp\FrameManager.ts:271:11)
    at CdpPage.mainFrame (C:\Fernando\Projects\whatsapp-test\node_modules\puppeteer-core\src\cdp\Page.ts:488:31)
    at CdpPage.evaluate (C:\Fernando\Projects\whatsapp-test\node_modules\puppeteer-core\src\api\Page.ts:2161:23)
    

[ultraman00]

This issue still persists in version in 21.3.8. puppeteer-extra 3.3.6
I dont use "--single-process".

    at assert (/node_modules/puppeteer-core/lib/cjs/puppeteer/util/assert.js:28:15)
    at FrameManager.mainFrame (/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/FrameManager.js:219:32)
    at CdpPage.mainFrame (/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Page.js:411:35)
    at CdpPage.evaluate (/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:872:31)

[incoming-th]

This still happen with puppeteer 21.6.0

I cannot share the code too as this is a SSO login page from Microsoft for a large company, but basically the browser crashes when the page loads the login form with:

Error: Requesting main frame too early!

This does not occur up to 21.4.1

[olex0r]

Confirmed, I have the same error and probably associated with iframes
Error: Requesting main frame too early!

[agata]

I've encountered a similar phenomenon. It occurs about once in every ten times. I am running Puppeteer version 21.5.2 on AWS Lambda. The process involves accessing three URLs twice each, and usually, this error occurs on the sixth and final access. I am accessing all of them using goto. Also, there is a process where I take about ten screenshots along the way.

[stale]

We're marking this issue as unconfirmed because it has not had recent activity and we weren't able to confirm it yet. It will be closed if no further activity occurs within the next 30 days.

[alexey-sh]

"puppeteer": "^21.9.0",
"puppeteer-core": "^21.9.0",

the error occurs randomly when I try to call page.url() after page.goto and 2500 ms timeout

[Tankonyako]

have same error, when i do 20 times goto, it giving this error and my process hang up

[OrKoN]

Are you awaiting page.goto to finish properly? if it's timing out or you are not awaiting it, the frame might not be there yet resulting in this error if you try to access it. Could anyone provide a minimal reproducible example?

[Tankonyako]

Are you awaiting page.goto to finish properly? if it's timing out or you are not awaiting it, the frame might not be there yet resulting in this error if you try to access it. Could anyone provide a minimal reproducible example?

i have one browser which 20 tabs (for example) then i open this tabs, and i have express server. /add route, when user reach this route, my code find free tab, open site, make screenshot, when i launch 3 tasks two times, after some 10-20 seconds it hang and show this exception, then page.url() crash app.
also if i do nothing, after some time - it crashed when i trying interact with page object, like get actual url()

[Tankonyako]

Are you awaiting page.goto to finish properly? if it's timing out or you are not awaiting it, the frame might not be there yet resulting in this error if you try to access it. Could anyone provide a minimal reproducible example?

i have one browser which 20 tabs (for example) then i open this tabs, and i have express server. /add route, when user reach this route, my code find free tab, open site, make screenshot, when i launch 3 tasks two times, after some 10-20 seconds it hang and show this exception, then page.url() crash app. also if i do nothing, after some time - it crashed when i trying interact with page object, like get actual url()

i also figured out, and with createIncognitoBrowserContext all works fine, i just create new context when need newPage, and closing context also with page.close().
i think when i use one browser to open 10 tabs for example, is using one cdp connection and sometimes it overflows, and giving this exception, maybe different context using own cdp.

[Tankonyako]

Are you awaiting page.goto to finish properly? if it's timing out or you are not awaiting it, the frame might not be there yet resulting in this error if you try to access it. Could anyone provide a minimal reproducible example?

i have one browser which 20 tabs (for example) then i open this tabs, and i have express server. /add route, when user reach this route, my code find free tab, open site, make screenshot, when i launch 3 tasks two times, after some 10-20 seconds it hang and show this exception, then page.url() crash app. also if i do nothing, after some time - it crashed when i trying interact with page object, like get actual url()

i also figured out, and with createIncognitoBrowserContext all works fine, i just create new context when need newPage, and closing context also with page.close(). i think when i use one browser to open 10 tabs for example, is using one cdp connection and sometimes it overflows, and giving this exception, maybe different context using own cdp.

also can confirm it happen only with headless mode, in non headless all works fine

[OrKoN]

If someone is able to provide a minimal reproducible example to confirm the scenario, it would be great.

[mshokoya]

I think I've found the solution.

I posted it on stackoverflow and I'll post it here

If you look in the puppeteer docs, it says a puppeteer instance is set to timeout in 30000ms (30seconds) by default.

puppeteer docs launchoptions

so when initialising the instance set the timeout option higher. I set mine to 31536000000ms (365 days)

const browser = await puppeteer.launch({timeout: 31536000000});

stackoverflow solution

[alexey-sh]

@mshokoya How to catch a possible error when launch is not launched within 30 seconds?

[mshokoya]

@mshokoya How to catch a possible error when launch is not launched within 30 seconds?

@alexey-sh I can think of 2 ways to catch the errors in puppeteer

1. launch puppeteer and put all your puppeteer logic in the same try/catch block e.g

  try {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('https://example.com');
    await page.screenshot({path: 'example.png'});

    await browser.close();
  } catch (error) {
    console.error('An error occurred:', error);
  }

2. use events. here are links to all events. but for the timeout error I think you'll need the page events. in it there are two events that look promising, the 'error' event and the 'pageerror' event. e.g

  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  page.on('pageerror', request => {
    console.error(`Request failed: ${request.url()}`);
  });

  page.on('error', request => {
    console.error(`Request failed: ${request.url()}`);
  });

  await page.goto('https://example.com');
  await page.screenshot({path: 'example.png'});

  await browser.close();

But for the 'Error: Requesting main frame too early!' error. It can be prevented by increasing the default timeout set by puppeteer. (I increased mine from the default 30 seconds to 356days). you can increase it by whatever you want, it could be 60 seconds, it could be 1 hour. e.g

  const browser = await puppeteer.launch({timeout: 31536000000, headless: true});
  const page = await browser.newPage();
  await page.goto('https://example.com');
  await page.screenshot({path: 'example.png'});
  await browser.close();

hope this answers your question

[shadaxv]

@mshokoya What you showed is not a solution to the problem at all, as the browser opens in less than 30 seconds.

[mshokoya]

@shadaxv after puppeteer launches ( i.e new page is open) the 30 second count down is started. after the 30 second countdown the page automatically closes due to the timeout being reached... if you then try to call a page method ( i.e page.waitForSelector() ) you will get the error

Error: Requesting main frame too early!

in other words the error occurs when you try to call a page method ( i.e page.url() or page.$ ) when the page does not exist

have you tried my solution ? did it not work for you ?

[shadaxv]

@mshokoya timeout is Maximum time in milliseconds to wait for the browser to start - to start not to run

protocolTimeout is set to 180 seconds by default, do you really think puppeteer would have a default time limit of 30 seconds to execute the script?

I have tried your solution and it is not working for me, in my case, the problem is irregular, and so far no solution I have tried has gotten rid of it

[mshokoya]

@shadaxv well it works for me. Ive tested it multiple times, if I leave it out, error occurs within 30 seconds. I set the timeout limit higher and I don't get the error. it may not work for you but it may work for someone else

[nodegin]

Is there a fix for this? it happens randomly with high probability which makes the automation nearly unusable..

[MathisZerbib]

Also looking for help on this one it append when I try to make async with puppetter when I launch other browser instances after closing the old ones, I use also puppetter-extra and I make use of waitForSelector

"puppeteer": "^22.6.3",
"puppeteer-extra": "^3.3.6",
"puppeteer-extra-plugin-proxy2": "^1.0.0",
"puppeteer-extra-plugin-stealth": "^2.11.2"`

[cage1618]

Same issue, any updates?

[andreabisello]

@nodegin the fix was released in these days, did you try to update? i tried, but because waitForTimeout was (correctly) removed i have to update my old testsuite in a lot of different places so until now i was not able to make a full run.

[charliefuller]

I was seeing occasional Requesting main frame too early! errors on 22.1.0. I applied #12225 as a patch on 22.1.0 and now see occasional Attempted to use detached Frame... errors instead.

Before:

Error: Requesting main frame too early!
  File "*****.js", line 65322, col 15, in assert
    throw new Error(message);
  File "*****.js", line 77371, col 32, in FrameManager.mainFrame
    (0, assert_js_1.assert)(mainFrame, "Requesting main frame too early!");
  File "*****.js", line 78997, col 35, in _CdpPage.mainFrame
    return this.#frameManager.mainFrame();
  File "*****.js", line 71199, col 29, in _CdpPage.goto
    return await this.mainFrame().goto(url, options);
  File "*****.js", line 811834, col 32, in exports.directHandler
    await page.goto(*****);

After:

Error: Attempted to use detached Frame '14424CDFADC9787ADAB520A75C4FC3A9'.
  File "*****.js", line 67191, col 19, in CdpFrame.<anonymous>
    throw new Error(message(this));
  File "*****.js", line 71199, col 41, in _CdpPage.goto
    return await this.mainFrame().goto(url, options);
  File "*****.js", line 811836, col 32, in exports.directHandler
    await page.goto(*****);

[OrKoN]

@charliefuller that makes sense, does the detached frame error indicate which page is detached for you? is the page perhaps crashed or you are using a stale page reference?

We do not have a reproducible example for this issue so it probably just pages that crash or stale references. With the assertion not firing it is now possible to catch it. If anyone has a reproducible example, please share it.

[charliefuller]

@OrKoN No but we only use one page. I don't think it's crashed or is a stale page reference.

[OrKoN]

@charliefuller do you have a reproducible example? is the page in the middle of another navigation/aborted/errored navigation when this happen?

[charliefuller]

@OrKoN It seems to happen randomly but when it does happen it's usually while waiting on await page.waitForFunction(...)

[OrKoN]

@charliefuller is it possible that sometimes waitForFunction keeps running while the page is navigating/redirecting? Perhaps, sometimes the waiting is done before the page somehow goes stale? it would be really great to get some example of what sort of interactions are happening. For example, we use Puppeteer in Chrome DevTools tests and we have not once caught this issue (same with our own tests) and we do a fair bit of waitForFunction/navigations on a single page. Are you running this on third party pages? is there a pattern in what sites it happens on?

[SABAR123001]

In my case it is happening for some particular session and also very rarely,other sessions are working smoothly . by making protocol Timeout can we overcome this error or do we have any other solution for this .

puppeteer version which i have is 22.8.0