It matters a lot when you are on a very tight budget. I can either get a Kaby Lake i3 for $119.99, or a FX-8320 for $129.99. It's really a no-brainer.
Another way to look at it: instead of putting an extra ~$100 on an i5, which has performance most people don't really take advantage of anyways, you could get a better graphics card.
