Maybe it is just because my eyes are worse than yours, my monitor has more pixels per inch or I keep my face farther away from the monitor than you do, but when I set a minimum font size, I want to set it to a value large enough to break some sites. (By "break" I mean the text runs off the right side of the screen or overflows into other elements.) But then there is no way to work around the breakage without tediously going back into preferences or about:config, lowering the minimum font size, then raising it again when I am done with the problematic site.
In other words, the reason I do not set a minimum font size is that Firefox treats the minimum font size as a hard limit to be observed even when the user explicitly asks to reduce the size of the text (which, again, I would need to do whenever the site does not render sanely at my preferred font size).
So I find myself increasing the size of the text many, many times a day. At least recent versions of Firefox allow me to add to the navigation toolbar a "[-|+]" widget that lets me increment or decrement the size with one click. Also, Firefox often remembers the text size for the next time I visit the same site.
Hope that answers your question.
I seem to recall a few sites' breaking (becoming difficult to navigate or fathom) if the text size is even slightly larger than the default size set by the site, so I am curious as to why you do not have that problem. (I seem to recall Amazon or Ebay being one such site.)
I never run into this problem due to having done so, but I see a lot of people complaining about it...