Your (and RMS's) definition of "freedom" and mine are different. I see software as "free" when it lets me do whatever with it, including modification and distribution with NO restrictions. GPL software adds restrictions, and even if those restrictions promote freedom, the restrictions themselves are the antithesis of free in my view.
Freedom is in the eye of the beholder, and often has many zero-sum components: the freedom you are requesting (to use someone else's work as part of your own without having to give anything in return) directly impinges on the freedom of others (as users do not have the freedom to modify the resulting binaries or devices that you are distributing, a freedom the GPL is designed to assure users have).
I encourage you to read this conversation I was in recently:
Thanks for that, I wouldn't have seen that (sub)thread otherwise. Basing an argument on the distribution of binaries seems sound -- even if some people seem to be able to look past the most well-structured and straightforward arguments no matter how things are formulated...
RMS writings tend to describe those extra features in your definition of freedow as power, because they give you control over others (with their consent when they accept the terms).
Of course, in a world where there was no proprietary software, the GPL's restrictions would not be very restrictive. That is kind of the point: to create such a world, or at least to create such an ecosystem.