Google historically splits off their "front end" which is dealing with messages from the internet and their "back end" which is processing what those messages want to do. It was briefly described in the original search paper they presented.
Of necessity, most machines that Google runs have 'private' (as in not directly addressable) addresses. This is how it seems that everyone runs things when they get above a certain number of servers.
Thirdly because one can assume that 'attacks' (whether they depend on XSS, overflows, or what not) are coming from the Interwebz its prudent to present the smallest possible attack surface on a machine which is being fed internet stuff. As you can imagine, those security constraints make for uncomfortable restrictions on what you can do in the 'front end', so a machine that is 'front end' has a different set of security constraints than a machine that is 'back end' (they are still constraints there of course but having gone through the filter of the front end machine which has greatly attenuated the possible exploit vectors).
I believe it is this armored 'front end' resource which is the scarce resource, but again, I've been gone over a year and things change quickly inside the 'plex.
Not trying to piss all over this, since I thought you might have been saying something I didn't know. But I have worked at Google for a long time and basically everything you wrote is incorrect and has been since at least 2005 before App Engine even existed (why would a paper written in 1998 be relevant?).
Without going into too much detail, it shouldn't surprise anyone that physical machines are interchangeable via cluster management software, so application front end and back end software all end up running on the same machines. They aren't addressed through DNS but by another mechanism. There is no special system administration for application-specific front ends vs back ends. Spinning up more instances of a particular server is a 1 line change. Reverse proxies (i.e. a non-app-specific front end) are also involved but they're not a bottleneck AFAIK (this problem is well "commoditized" in open source whereas other parts of Google's stack are not).
I would be curious to see a comparison with competitors like Heroku, which I've heard are pretty expensive too. Theoretically Google should be cheaper because Heroku runs on AWS and thus is paying for Amazon's profits (I have no idea how they compare.)
The original point is "almost right". The scarce resource reflected in the new pricing model is not machines, but RAM. The old pricing model assumed CPU time was the essential resource. The new pricing model bills you for the amount of time your instance (which has a fixed RAM reservation) stays resident.
It sounds plausible that "on demand" instances i.e. search servers are a bottleneck, while bulk offline processing (in the back-end) i.e. PageRank calculations is not such a constraint.
Yes, those offline processing instances could be suspended while dealing with a traffic peak, but peak traffic is still often a challenge, especially now that Google is pushing more and more stuff onto the web servers (Instant).
That might be what he meant.
Disclaimer - I don't work for Google, it's just a guess.
Of necessity, most machines that Google runs have 'private' (as in not directly addressable) addresses. This is how it seems that everyone runs things when they get above a certain number of servers.
Thirdly because one can assume that 'attacks' (whether they depend on XSS, overflows, or what not) are coming from the Interwebz its prudent to present the smallest possible attack surface on a machine which is being fed internet stuff. As you can imagine, those security constraints make for uncomfortable restrictions on what you can do in the 'front end', so a machine that is 'front end' has a different set of security constraints than a machine that is 'back end' (they are still constraints there of course but having gone through the filter of the front end machine which has greatly attenuated the possible exploit vectors).
I believe it is this armored 'front end' resource which is the scarce resource, but again, I've been gone over a year and things change quickly inside the 'plex.