I use a Firefox addon called NoSquint,[1] which lets me set zoom levels globally and per-site. It's one of the most useful Firefox addons I have, and really makes my browsing experience much more pleasant.
I also used to use an addon called Stylish[2] to force a nice CSS style on websites. I've since switched to using a Stylish stylesheet I like directly with Pentadactyl[3], which gives me the same effect of using Stylish but without needing an extra addon (since I'm using Pentadactyl for its other features anyway).
Using the above combination standardizes the look of virtually every website I go to and makes it a pleasant experience for me, where both the font sizes, font colors, and background colors are all quite pleasant and readable.
Unfortunately, this does mean I don't get to see occasionally pleasant original website designs, but it's a sacrifice I'm more than happy to make for a painless websurfing experience.
That is indeed simpler, but not nearly as powerful or as flexible.
As far as I know, no default Firefox setting will allow you to impose different font sizes per website. Also, the setting you mention will change only the font size, and not the zoom level as a whole (NoSquint can change either).
Finally, the setting you mention does not affect font colors or background colors, as the Stylish stylesheet does.
I also used to use an addon called Stylish[2] to force a nice CSS style on websites. I've since switched to using a Stylish stylesheet I like directly with Pentadactyl[3], which gives me the same effect of using Stylish but without needing an extra addon (since I'm using Pentadactyl for its other features anyway).
Using the above combination standardizes the look of virtually every website I go to and makes it a pleasant experience for me, where both the font sizes, font colors, and background colors are all quite pleasant and readable.
Unfortunately, this does mean I don't get to see occasionally pleasant original website designs, but it's a sacrifice I'm more than happy to make for a painless websurfing experience.
[1] - https://addons.mozilla.org/en-us/firefox/addon/nosquint/
[2] - https://addons.mozilla.org/en-US/firefox/addon/stylish/
[3] - https://addons.mozilla.org/en-Us/firefox/addon/pentadactyl/