I'm personally not going to A/B test larger webpage downloads for my customers.
Smaller is better & faster. There's no noticeable difference in image quality.
So my A/B test is 4 months before vs. 4 months after. This is not a small sample size. It's around 500k uniques on each end. 3M+ page views on each end.
I've also broken & analyzed the traffic down by channel, new vs return, and any other way I can to attempt to remove any false positives. From all the data, the large increase in conversion was because of this particular change.
I have been aware of "holiday shopping bias" around nov / dec, but January is normally a very shitty month for conversions. January / Feb our conversion was higher than any previous month before this as well.
This is actually why I've waited to Feb to really solidify the results in my own mind. And that's why there's 4 months of data to back it up. Every day since the change, being better than before.
I don't need to A/B test this one. The savings on my homepage was 4MB. We're not talking small bytes here. This is a major issue with regards to anyone using Shopify.
> I'm personally not going to A/B test larger webpage downloads for my customers.
That's very sensible. But then you can't state that smaller images alone increased your conversion rate by 56%. Maybe your products are just more Winter-appropriate? If you can't control for confounding factors, you can't make a strong conclusion.
...On the other hand, in a contrived example, one could imagine smaller images could increase conversion rate by 1000% (if it cuts several minutes off the process; a website that takes 5 minutes to load will get no customers).
What I would like to see is an analysis of how much faster the shopping experience is post-compression. Perhaps for some of those edge cases (like your 250+ item shoppers)
My worst day after this change, was better than my best day previous to this change. Thousands of visitors.
This trend has continued for 4 months after the change. Hundreds of thousands of visitors.
I'm the only dev on this project and I know what changes day to day.
Can full 56% be attributed? No. I added opt-in monster with 10% & 20% (A/B) discount in Feb which boosted conversions. 10% actually out performs 20%. Go figure.
Can more than 0% be attributed to smaller images. Yes. So I'd say this change helped between 0% and 56%. Either way, it's statistically significant.
4MB savings on homepage going to save you from a shit tonne of bounces and help move more people towards that checkout page.
Smaller is better & faster. There's no noticeable difference in image quality.
So my A/B test is 4 months before vs. 4 months after. This is not a small sample size. It's around 500k uniques on each end. 3M+ page views on each end.
I've also broken & analyzed the traffic down by channel, new vs return, and any other way I can to attempt to remove any false positives. From all the data, the large increase in conversion was because of this particular change.
I have been aware of "holiday shopping bias" around nov / dec, but January is normally a very shitty month for conversions. January / Feb our conversion was higher than any previous month before this as well.
This is actually why I've waited to Feb to really solidify the results in my own mind. And that's why there's 4 months of data to back it up. Every day since the change, being better than before.
I don't need to A/B test this one. The savings on my homepage was 4MB. We're not talking small bytes here. This is a major issue with regards to anyone using Shopify.
See if you can guess which month the change happened on this chart: http://i.imgur.com/S95EX9Q.png
Because I can see it.