I mean, the app is meant to help organize your contacts, is it not?
If I see an app ask for my contact data without good reason, I'm immediately out in most cases. But they've got a good reason: it's what the app is supposed to do...
The app is "a personal CRM that reminds you to keep in touch". There are people that I want to keep in touch with (which I can add manually), and the rest of my contact list, who don't need any organizing.
