Or uses stripe which hides your real card number from the merchant. The annoyance of adding a card is one thing but fear of an app developer getting hacked and exposing your card doesn't strike me as a very strong reason.
As an app developer, most users also seem to think developers are to blame for issues like the App Store failing to download apps, double-billing etc, so I don't think buying though Apple makes much of a difference there.
That's flawed logic. Your users are telling you they can't tell apart merchant/platform so you should build on top of that and assume they won't know that you're not PayPal
Pretty much everyone takes PayPal which shields you from this