Total Aside. As a compiler and runtimes guy, I'm super excited for streaming compilation. I think stuff like this and ethereum for distributed computation is really cool stuff! :D
Streaming compilation is the way it was always historically done. One reason is that computers used to not have enough RAM to store whole non-trivial programs in it in AST or another intermediate form.
Second reason is that this approach matches how the underlying theory of languages and automatons works. One can view modern AST producing compiler frontend as compiler that compiles it's input into program that builds the resulting AST.
On the other hand many modern optimalization passes simply cannot be done in streaming manner or even by any pushdown automaton.
