In a sense, yes - to build up your muscles they must first be broken down, a cycle which is going to create some amount of stressful churn. If you think about any machine, a higher operating energy/intensity generally correlates to faster breakdown.
At the same time, humans in good shape are definitely healthier, and too much calorie restriction can lead to malnourishment.
There's a sweet spot somewhere, proponents of calorie restriction are basically saying to err on the side of eating less.
I think it's the sort of activity that matters. Low impact exercise wears on your joints/etc less than higher intensity stuff (think swimming or walking compared to running).
At the same time, humans in good shape are definitely healthier, and too much calorie restriction can lead to malnourishment.
There's a sweet spot somewhere, proponents of calorie restriction are basically saying to err on the side of eating less.