Then press the button with a handset instead of the one with a camera; it's called "FaceTime Audio" for a reason.
If you had added them to Favourites and are calling from there, you added FaceTime (video) as the favourite; delete that shortcut and re-add them to Favourites as FaceTime Audio.
You're assuming I use FaceTime on a phone. I do not. To clarify, I'm describing the experience of using the Contacts app in MacOS 10.14.6 to place phone calls.
If you had added them to Favourites and are calling from there, you added FaceTime (video) as the favourite; delete that shortcut and re-add them to Favourites as FaceTime Audio.