How many flags would you use for English? If you want to use flags of the most populous countries where English is the primary language, the second country after the US is Nigeria (182 m people).
Well, how many languages have flags? There are thousands of languages, but hundreds of countries. Surely, some languages would be left without a flag?
And this doesn’t affect just some "obscure" languages. For instance, what flag would you use for Arabic, an official UN language? What language does a flag of India, most populous language stand for?
Because about two dozen countries use $ as their currency symbol and another half dozen or so $ with another letter or two and not all of them have English as an official language (or an only technically not official language like the us).
(Re: colors: see my suggestion to use these colorful icons to represent time zones)