1. This was created by a Microsoft employee (a "Microsoft Developer Evangelist"). Maybe this doesn't matter to you, but considering the $100m marketing budget and the widespread use of astroturfing, it seems a little dishonest to me (certainly treading in a grey area) that there isn't full disclosure. YMMV
2. As someone who cares deeply about research methods, I'd argue that there is little to no value in superficial, subjective and contextless comparisons like this.
Edit: He has now added a statement that he works for microsoft. It was not there before I posted this comment
I didn't say, hint or imply that it was listing inaccurate results. I stated that it's superficial, subjective and devoid of context. Frankly, it means nothing.
For example, someone searches for "bongo". The forth results are a radio station, a page about the animal and an email/calendaring system. Given those results, someone votes for one. Why? We don't know. They might barely now. One thing we do know: it was a one-off search anyway, not something that was part of someone's normal workflow.
I strongly disagree. Incidentally, while this may be new since your first message, there is disclosure on the landing page which I found quite adequate. I notice Google leads by a significant margin. Is Bing looking better than it really is? Who knows, but while I don't plan to switch away from Google, Bing is OK by me - the quality of results are pretty good and its fast. Yahoo not so much (and I got no Yahoo results to my tests at this site) but I'd given up on Yahoo search before Google came along.
You assume it's a one-off search and that the users are simply licking randomly on whatever they see that's immediately interesting. I see no justification for this assumption. Personally, I did what I did with any new search engine: perform 10 searches on topics ranging from topical to deeply obscure, and look at the first 5 results for each search to evaluate the quality of both the returned sites and the text clips for each.
If this was the subject of a paper submitted to the journal of information processing, you'd have a valid point. But as a 'what happens if...' experiment on the web, I'm just fine with it.
the context seems pretty obvious, to let people compare the different search engines without being biased that they are using a search engine other than Google
You are misusing the word "context" - http://www.answers.com/context : 1. The part of a text or statement that surrounds a particular word or passage and determines its meaning.2. The circumstances in which an event occurs; a setting.
The searches here are without context, the rating of results are without context. All we know is that on arbitrary, one-off searches, a set of people who may or may not be representative of anything clicked the button above a certain set of results for some unknown reason.
<i>2. As someone who cares deeply about research methods, I'd argue that there is little to no value in superficial, subjective and contextless comparisons like this.</i>
I care deeply about research methods too, but I don't understand why this is bad. If the goal is to know which search engine is best -- a subjective question in the first place -- it's gathering judgments from people on exactly that topic.
Of course, you can't take the statistics collected overly seriously, since it's hard to know how seriously different people are taking it and such. But it's probably more true than not.
2. As someone who cares deeply about research methods, I'd argue that there is little to no value in superficial, subjective and contextless comparisons like this.
Edit: He has now added a statement that he works for microsoft. It was not there before I posted this comment