While interesting, performance is a tiny concern here. Most browser automation is going to have a lot of implicit and explicit waits that dwarf the differences in performance between libraries. Moreover, someone may choose Puppeteer over Selenium for functional differences (for example, Selenium is easier to catch and block)
It’s only a “tiny concern” because the results seem so similar. But if one was drastically slower, then it is important.
However I don’t believe this testing methodology shows the true performance impact of the different frameworks. Only once you have a significant number of assertions and page interactions then to we really see the performance impact of the architectural differences.
Sure. In my experience (I've done a ton of this kind of work) well designed flow can take at least 30 seconds, up to several minutes depending on what you're automating. A difference of 5-10 seconds tends to not matter in that situation, and you'll be more interested in things like code ergonomics, memory leaks over time, etc.
It is indeed hard to have an all-encompassing comparison that will take everything into account. Unfortunately, to have precise answers in feasible time you have to narrow the scope. Here we looked at one parameter of a complex equation.
