The point is, you wouldn't know unless you have complete understanding on every aspect of your device and every bit of the software.
Nobody would be installing a Linux kernel and use the phone like that, they would be installing a distro. There are so many vectors of attack, the person who puts the distro together doesn't need to have malicious intent, the supply chain could be compromised.
