I do that too. I even got an app that does this automatically. I just get a silent notification afterwards, it's great. It's very rare that I'm expecting a call from someone I don't know already, so there's no problem.
Unfortunately, not everyone has this privilege -- people who work with sales, for instance.