The main issue with caffeine, however, is that it is very bitter and so most caffeinated beverages come with a large amount of sugar to mask the taste.
When I started drinking coffee regularly in college, I decided I would only allow myself to drink it black to avoid sugar and milk (I can't digest lactose).
It didn't take long at all before I preferred it that way. Now I love a back espresso with no sugar.
Same with tea, for me. I drank tea and tisanes with sugar and milk when I was young. Then over time I went to just sugar, and then plain. Now I prefer tea/tisane/coffee plain.
The main issue with caffeine, however, is that it is very bitter and so most caffeinated beverages come with a large amount of sugar to mask the taste.