Not a silly argument at all. Every time we say, "lets spend $x on y", it is fair to question whether that money can be used to feed and clothe... or defend.. or fix the infrastructure, etc.
Of course it can, there's no reason it couldn't. That doesn't make it a proper challenge. One would be "instead of $4B, $X should be invested on this, because of such and such". X == 0 without details is a big red flag.