Google did try this. I don't think it ever left the lab, but I used it for a while. I think they may have canned it; I just realized I haven't seen it in a while. It was called searchwiki. I think it was originally just supposed to apply to your own results, but I'm sure they analyzed the tendencies to see if it would've helful to use overall.
Honestly, such a system would be ridiculously easy to game with a botnet, so there needs to be significant work in the area beforehand.
I think the 'X" was live for maybe a month or two. I figured either or both that there was a resource/algorithm problem that prevented them from continuing in a fruitful way, or that it was a bit of user survey they could use in aggregate as preference weighting on the indexes/results. That is, not as a function of personal filtering, but to see if there were commonalities in what people considered to be bad sites. This may be further supported by the fact that the lifetime of the feature may have been too short for gamers to exploit it.
The problem with that, is again, spam. How do you differentiate fake votes from real ones? You can do per-IP limiting, and probably other things I'm not aware of (please comment about them), but it's still pretty open to abuse I'd say.
Google has proven to be very effective data collectors (see "What they know" stuff from WSJ e.g. http://online.wsj.com/article/SB1000142405274870330970457541...) - it may on the one hand be surprising they don't use this data more effectively to identify and thwart spam. OTOH, they're making a lot of money on spammy SERPs.
I'd even be OK with "Thumbs Up/Down this result just for your own personal book-keeping", so that you can promote/demote results in your own search history (try searching for "_____ tutorial photoshop" and you'll get tons of splog links before you eventually hit a golden result. It'd be nice to save time later).
Google briefly had a feature like this in their results, but they've removed it. All that's left is the Star system, which isn't quite the same.
Did this page help you find what you were looking for? Was this page useful?
etc