I'd guess that the clickstream data can yield conclusions that are nearly as accurate as manual voting, if not moreso. It's not worth cluttering the interface for data with such low marginal value.
Using clickstream data for defining ranking is a must, but not enough.
Manual rating adds value to web site rankings.
Google Toolbar's data adds even more value to the ranking.
I'd say all that clicking data should be affect search rankings even more than hyperlinks.
Assuming that people actually use the "down" buttons, I can envision visiting a spammy link and wanting to go back to the search results page and kill a result.
