That's not true; there is a very, very specific alert shown before Facebook accesses any phone numbers. I'm not making this up, it's just how it works. Facebook does not get anything "whether you like it or not". They get the information if you specifically allow it, and only if you try to enable the Sync feature.
