I was wondering if it was toxic or non-toxic. Silica gel packets say "do not eat" but they're in a ton of our food packaging so I'd hope they're non-toxic.
Food-grade additives are about avoiding adulterants, etc. I can't imagine people are going to that kind of effort for cheap hygroscopics.
Also, and here's what I suspect the real reason is; silica gel sold as a dessicant is hard (not quite as hard as silica glass, but hard), in relatively large particles, and indigestible. That's mechanically bad news for your intestines.
Would be easy to investigate. Have desert people more cysts in the GI tract than other people? Digestive tissue is being distroyed and sheding constantly on the other hand, so probably not.