"Let large quantities of data solve your problems" might not be the best advice if you are hardware constrained. Not everyone has the petabytes of storage and the terabytes of RAM that Google has.
I guess cutting edge natural language apps are going to be the playground of the big boys until PCs reach the scale necessary to do experiments.
Practical natural language apps might require much less, though: see Norvig's spell checker for example. You can probably fit the google index from 1998 on a single modern machine.
I guess cutting edge natural language apps are going to be the playground of the big boys until PCs reach the scale necessary to do experiments.