And part of US Defense spending helps defend countries like Switzerland. Maybe we still spend too much, but we also "defend" a lot more countries than ourselves, and a lot more territory than Switzerland.
Switzerland is a neutral country and not part of NATO. The US has no obligation to defend them. Almost every adult male Swiss citizen has received military training. The country may not spend as much as the US, but Swiss citizens are willing to pay for their freedom in a different way.
(..and yes, only men are conscripted. Switzerland is a great country, but there continue to be problems with sexism, even today)