I was meeting a co-worker who introduced herself as a "New Yorker" (from Long Island) and I told her about how a there were a lot of New Yorkers here, like one of my close friends from Syracuse.
She gave me basically the same patient response I got here about how upstate was different. So I'm trying to be more conscientious about how I use that particular demonym and the context of it. But truth be told, I tend to throw most people from the north into a pretty broad "Doesn't like sweet tea and sometimes might talk loudly" category - so I'm still not able to guess if someone is from a different part of NY unless they tell me.
Upstate New Yorkers also works fine as an alternative.
Michigan has it easy with their conveniently hand-shaped state. NYS but not NYC is a pain to describe.