Search quality is diminishing because "the web" as a graph of hyperlinked text documents where any domain name is a priori equally likely to contain a gem of information you're looking for, doesn't really exist anymore.
It does, Google just heavily penalizes non-top-website pages for some inexplicable reason, sometimes outright de-indexing them. I guess you could argue this is a consequence of stopping spam - better to just ignore all small sites rather than risk catching blogspam. But somehow there's still plenty of blogspam.