Skip to content

Latest commit

 

History

History
84 lines (47 loc) · 8.4 KB

why-rod.md

File metadata and controls

84 lines (47 loc) · 8.4 KB

Why Rod

There are a lot of great projects, but no one is perfect, choose the best one that fits your needs is important.

Compared with other libs

Chromedp

Theoretically, Rod should perform faster and consume less memory than Chromedp, the architecture of Rod is more stable and consistent.

Chromedp uses the system's browser by default, it can cause issues if you accidentally upgrade the browser. Chromedp uses different Go module to manage devtools protocol version which can cause issues like this. Each Rod version will ship with a specific Chromium version and its devtools protocol version with full unit test against them.

Chromedp uses a fix-sized buffer for events, it can cause dead-lock on high concurrency. Because Chromedp uses a single event-loop, the slow event handlers will block each other. Rod doesn't have these issues because it's based on goob.

Chromedp will JSON decode every message from the browser, rod is decode-on-demand, so Rod performs better, especially for heavy network events.

When a crash happens, Chromedp will leave the zombie browser process on Windows and Mac.

Rod is more configurable, such as you can even replace the WebSocket lib with the lib you like.

For direct code comparison you can check here. If you compare the example called logic between rod and chromedp, you will find out how simpler rod is.

With Chromedp, you have to use their verbose DSL-like tasks to handle code logic. Chromedp uses several wrappers to handle execution with context and options, which makes it very hard to understand their code when bugs happen. The heavily used interfaces make the static types useless when tracking issues. In contrast, Rod uses as few interfaces as possible.

Rod has fewer dependencies, a simpler code structure, and better test automation. You should find it's easier to contribute code to Rod. Therefore compared with Chromedp, Rod has the potential to have more nice functions from the community in the future.

Another problem of Chromedp is their architecture is based on DOM node id, puppeteer and rod are based on remote object id. In consequence, it's not only slower and also prevents Chromedp from adding high-level functions that are coupled with runtime. For example, this ticket had opened for 3 years. Even after it's closed, you still can't evaluate js express on the element inside an iframe. Besides, Chromedp maintains a copy of all the nodes in memory. It will cause race condition between local NodeID list and DOM.documentUpdated, which can cause confusing issues like #762.

Puppeteer

Puppeteer will JSON decode every message from the browser, Rod is decode-on-demand, so theoretically Rod will perform better, especially for heavy network events.

With puppeteer, you have to handle promise/async/await a lot, it makes elegant fluent interface design very hard. End to end tests requires a lot of sync operations to simulate human inputs, because Puppeteer is based on Nodejs all IO operations are async calls, so usually, people end up typing tons of async/await. If you forget to write a await, it's usually painful to debug leaking Promise. The overhead grows when your project grows.

Rod is type-safe by default, and has better internal comments about how Rod itself works. It has type bindings for all endpoints in Devtools protocol.

Rod will disable domain events whenever possible, puppeteer will always enable all the domains. It will consume a lot of resources when driving a remote browser.

Rod supports cancellation and timeout better, this can be critical if you want to handle thousands of pages. For example, to simulate click we have to send serval cdp requests, with Promise you can't achieve something like "only send half of the cdp requests", but with the context we can.

Playwright

Rod and Playwright were first published almost at the same time. Most comparisons between Rod and Puppeteer remain true to Playwright, because both Playwright and Puppeteer are maintained by almost the same contributors.

As Playwright stated on their doc "Playwright enables reliable end-to-end testing for modern web apps.", the focus of the project is testing. But the focus for Rod is more general, for both web automation and scraping, which make the design focus more on flexibility and performance.

One of Rod's architectural goal is to make it easier for everyone to contribute and make it a pure community project, that's one big reason why I chose Golang and the MIT license. Typescript is a nice choice but if you check Playwright's design choices, any and union types are everywhere, if you try to jump to the source code of page.click, d.ts files will let you understand the reality of typescript. Golang is definitely not good enough, but it usually introduces less tech debt than node.js typescript, if you want me to choose which one to use for QA or Infra who's not familiar with coding to automate end-to-end test or site-monitoring, I would pick Golang.

Their effort for cross-browser support is fabulous. But nowadays, HTML5 is well adopted by main brands, it's hard to say the complexity it brings can weight the benefits. Will the cross-browser patches become a burden in the future? Security issues for patched browsers is another concern. It also makes it tricky to test old versions of Firefox or Safari. Hope it's not over-engineering.

Selenium

Selenium is based on webdriver protocol which has much less functions compare to devtools protocol. Such as it can't handle closed shadow DOM. No way to save pages as PDF. No support for tools like Profiler or Performance, etc.

Harder to set up and maintain because of extra dependencies like a browser driver.

Though selenium sells itself for better cross-browser support, it's usually very hard to make it work for all major browsers.

There are plenty of articles about "selenium vs puppeteer", you can treat rod as the Golang version of Puppeteer.

Cypress

Cypress is very limited, for closed shadow dom or cross-domain iframes it's almost unusable. Read their limitation doc for more details.

If you want to cooperate with us to create a testing focused framework base on Rod to overcome the limitation of cypress, please contact us.

What does Rod mean

Rod is the name of the control device for puppetry, such as the brown stick in the image below:

rod

The meaning is we are the puppeteer, the browser is the puppet, we use the rod to control the puppet.