Put me in the opposite camp: I'm young and nearsighted, which means my close vision is very good. I usually read HN on my iPhone at min zoom, and I find it really frustrating when mobile sites helpfully won't let me zoom out to read more than a paragraph at a time. (In fact, I found hitting ā- a few times made this post much more readable.)
Your complaint is really the same: designers are imposing their will over users, in a manner which ultimately negatively affects user experience.
I find myself frustrated in both directions: sites whose default font is too small (and isn't resizable), and those whose default fonts are too large, and non-resizable. To say nothing of color choices which negatively impact readability.
