Not anymore. Not on the M1s. They're far and away faster than anything but the beefiest desktop. A lua parser benchmark I ran in a VM on my M1 beat my 5950X by a factor of two.
The only remaining limitation are missing some ARM docker containers (which is slowly being solved) and the machines not being suitable for ML (in which case you need a desktop or to rent server time on the cloud).