I think the problem is that most of the data they have on me is implicit in the data they have stored about other people. They definitely have my phone number, but I'm not sure they store the result of a computation they could easily do on demand with what effectively would be a join statement (no ML required), namely reconstructing my call history with everyone that has the facebook app installed. Again I'm not sure if the GDPR has any provisions against that, because it affects two parties one that has consented to the storage and another that hasn't. Also it is not clear to me how an average citizen would be aware of the fact that even if they only have your phone number stored they are most likely able to reconstruct most of your call history as well.
> namely reconstructing my call history with everyone that has the facebook app installed
IANAL, but I think that would be okay, as long as they don't associate any/strip all personal identifiable information from your phone number, and don't store the phone number in plaintext.