Even without verifying safety numbers, you’re still better off on Signal then you would be on another platform that doesn’t even offer the option of verification. If you’re looking to MITM a conversation on Signal you can only guess whether or not the recipients have verified each other, whereas on a platform like iMessage you know they haven’t because it’s not an available option.
SS7 spoofing is not a hard thing to do. Who knows if you really are initiating to +14055551212 who you think they are. I guess using multiple techs in serial could obfuscate the initiation correctly (voice, IM, Social Media, etc)
PGPFone had a neat thing where it'd show each participant of a voice call a short string they'd read out loud, and then the crypto would use those for handshaking. MITM'ing voice, as part of a freeform conversation, especially between friends, is a lot harder.