Reply to wizofaus: 9 minutes is more than much more than enough to revert, at least in our setup. Specially if some automated alert triggers, then we could revert well within a minute.
That’s what staging environments are for : deploying every commit to check everything is fine, doing the QA and get the alerts.
Doing this in production is being disrespectful of your users. Users are not waiting for your PRs to be approved and merged, they can be as delighted by your new button’s shade of lightgrey if it is released some days (even weeks !) after.
That's my feeling, that you get all the benefits of rapid/frequent deployment by doing so to non-production environments, and deploy to production when you have a good reason to do so.
But either way I don't really understand being driven by a desire to ensure every commit results in a deploy. It boggles my mind that a complex product, especially involving mobile platforms, could be fully built, ci-tested, deployed then e2e tested within 10 minutes, even with a massive array of distributed computing power. Mostly due to 3rd party tools that give you no choice but to wait 20 minutes or more for certain steps to complete. I'm intrigued to know how it's done.
More advanced/modern tool chains like k8s can help; rollout can be done in minutes. Also makes it easier to do blue/green deploys if you’re serving high throughput services where you can get an error signal from upgrading a small fraction of your service workloads.
