I don't think aptitude is the reason. A lot of people are physically unfit and have traumatic sports education experiences, probably even more so than math, but athletes remain stars.
The issue is entirely cultural. Entrepreneurs, celebrities, socialites are venerated, scientists aren't. There's no mainstream cultural recognition of scientists and science itself, at best people get attention who monetize research.
The "issue" is entirely about money. Sports make so much money because so many have fun watching them. And it's an issue only if you want fame so badly as to make an issue of the fact that what you do is anonymous.
As a European who was unfit for sports, the social stigma is much worse. It is socially acceptable to be bad at maths, but you are looked down upon if you are overweight, lack coordination or are otherwise unfit.
For me it was a self enforcing effect where it also impacted how I viewed myself and I did not feel confident enough socially.
Obesity is one of the few remaining social stigmas that are "acceptable" to joke about. I don't think the average person know how it makes you feel when people passing you in the street look at you and make comments or try to avoid you.
There are so many micro aggressions that happens where each stare, laugh or comment just breaks you down mentally and isolates you socially.
"It is socially acceptable to be bad at maths, but you are looked down upon if you are overweight, lack coordination or are otherwise unfit"
Yes, that is sadly often the case.
My advise if you want to change something, find acticities that are fun. Everything with water is good, if you are overweight, but it seems, you likely would not like the public swimming pool. But there are offerings for water exercises especially for obese people. Being in a group helps ignore what others think. Otherwise there are hidden lakes for example. (Or get a private swimming pool if you have the funds)
The issue is entirely cultural. Entrepreneurs, celebrities, socialites are venerated, scientists aren't. There's no mainstream cultural recognition of scientists and science itself, at best people get attention who monetize research.
Just compare how many people know Musk but don't know who Tom Mueller is. (https://en.wikipedia.org/wiki/Tom_Mueller)