Not necessarily -- trademark infringement requires an element of deception or confusion, not just any usage of the mark.
If an OEM sold a phone with a stock AOSP build, I think they'd be fine with calling it "Android" without permission. Google might attempt to enforce their mark anyway, but they wouldn't have much of a case.
If the phone ran a fork like LineageOS, the OEM would need to be more careful about their use of the mark, but I imagine they'd still be okay with language like "Android-based".
