Deciding not to AB test because of this article would be a huge mistake.
Learning from mistakes made by others, and avoiding them is what I would suggest as the take away.
I work for a successful ($50M+ revenue) bootstrapped startup. And one of the reasons for the success is that AB testing became part of the company's culture, as soon as there was enough data coming in for the tests to become useful.
AB testing is so important that we have built our own in-house framework that automatically gives results for our company specific KPIs.
Learning from mistakes made by others, and avoiding them is what I would suggest as the take away.
I work for a successful ($50M+ revenue) bootstrapped startup. And one of the reasons for the success is that AB testing became part of the company's culture, as soon as there was enough data coming in for the tests to become useful.
AB testing is so important that we have built our own in-house framework that automatically gives results for our company specific KPIs.