The benefit of using code is that you have to know what you are doing. Having experienced what happens when people build data driven systems out of building blocks, with out a thorough understanding of what they are doing (brittle, failing is strange ways under load, general unreliability and low quality) I am suspicious
At least for testing, and traditional automation (aka code) the bar is knowing what you're doing AND knowing the product enough to be able to test it effectively.
We remove the code requirement, making testing accessible to more folks - i.e. product managers and product designers who have great knowledge of the product, but don't want to or can't code. Also, this doesn't tend to exclude developers either. Currently our user base is roughly 1/3rd engineers, 1/3rd product managers/designers, and 1/3 QA folks.
Completely agree that testing should be more accessible to product managers and designers, and love the concept. Consider the simple example of a web form with multiple `input` elements - if I use your product to click on each input and configure a QA procedure, how does the system unambiguously identify each input given that the page layout may change in the future?
The current code-based testing frameworks force me to add an unambiguous marker to the `input` element, like an attribute or ID, which also makes it easy to query from the DOM during the QA process. How does this QA product handle breaking changes to the UI, and how robust could you expect it to be to code changes?
It identifies things visually, and optionally with OCR as well. If it moves likely no issue. Changing shape or size too much, as well as not matching OCR can cause failures - as expected. If it is an expected update and the test needs updating, we smartly suggest updates to the target if we can find one. Alternatively we have a crowd-based service to help write abs maintain your tests; it’s usually used by teams needing high leverage when managing a lot of tests.
Agree, no-code seems to optimize for a large number of people being somewhat effective, but with less control over the output (increasing fragility, rework)
Code is more upfront effort but more control and thus less fragility but more maintainable over time (if done right).
I imagine there are some/many situations where throwing many people at a problem is the “best” way and this would suit that quite well I guess.
Playwright is excellent too. Playwright is much more forgiving with tests that need to high different origins (common with Enterprise apps) and multiple browsers in the same test (to verify collaborative editing etc). If you're considering Cypress, I'd highly recommend also giving Playwright a look (https://playwright.dev )
Agreed; also, even with human testing, their nuance can be a double-edge sword. Testing, and specifically QA is hard - as today it's mostly about the "assurance" part; does one feel assured enough to ship this, which is subjective.
The benefit of using code is that you have to know what you are doing. Having experienced what happens when people build data driven systems out of building blocks, with out a thorough understanding of what they are doing (brittle, failing is strange ways under load, general unreliability and low quality) I am suspicious