Individuals from China (a big new buyers market) to India (jewelry) to the US that buy gold, do in fact hold it physically.
Some large scale investors of the Paulson type do hold huge sums in GLD, but they're not your typical gold owner, and they're dwarfed by the smaller physical gold market transacted between individuals.
Americans own more gold than the US Government / Fed does, and they're not holding it in GLD ($64 billion value).
Your typical gold owner has less than $100,000 worth of gold, and holds it physically.
You buy the gold from a bank, but in most cases, especially if the gold is purchased for investment, you will not hold it physically.