If you distributed binaries made from GPL code to me, you're obliged to provide the source code if I ask you. You can do that by pointing me to an upstream url fine, but you need to be sure that that url will be valid when I ask you.
Even beyond that, we need to be able to prove that the binaries we're distributing were made from the source that we claim they are, and be able to provide exact instructions for a customer to independently re-create the binaries. That usually means that we stick to versions of libraries and compilers that are on the GA installation disks for a system, with exceptions made for a few security-critical libraries like OpenSSL.