Same where I grew up. The town regularly won awards for far surpassing all the drinking water standards, even the super strict totally voluntary standards they additionally attempted to meet and exceed.
Then I moved to Houston. After talking with some water treatment engineers, I only drink bottled water here. Was very hard to change my habits.
I don't know about the health side, but growing up on South Australian rain and tap water, I think the Sydney tap water tastes absolutely foul by comparison.
Tap water in Sydney and most of Australia is excellent.