On the other hand, if you hate working out and working, and you have a limited amount of willpower, you might end up spending all of your willpower working out.
True. But you're probably exerting yourself too hard if a workout session taxes you that much (or you have it built up in your mind so you'll hate it no matter what). Remember Worf's exercise of holding 13-pound weights straight in front of him? To do that, you don't even start with weights, you just hold your arms out. There's no shame (literally none, this is actually the secret of developing an exercise mentality/physiology) in lifting incredibly light things, even just your arms.
I do hear you, willpower is precious. But since exercise helps create willpower, too, and stress mitigation, it's a highly lucrative investment.
A lot of people just hate things associated with exercise, like going to the gym with a bunch of people, or running around their neighborhood. You can always start by doing bodyweight exercises like pushups, pullups, crunches, and yoga. If you still hate it, you could do what many Americans working menial jobs do and get high beforehand.