I have also done both and I'm pretty certain higher maths is harder. Its not just motivation. Most people are frightened of leetcode style interviews, which require just an undergraduate level understanding of algorithms. There's plenty of motivation to learn it but most people can't. Higher maths or physics requires way more intellectual horsepower than that.
