I once used a crowdsourcing system called CrowdFlower for a pretty basic task, the results were pretty bad.
Seems like with minimal oversight the human workers like to just say they did the requested task and make up an answer rather than actually do it (The task involved entering an address in Google maps, looking at the street view and confirming insofar as possible if a given business actually resided at the address in question, nothing complicated)
Edit: woops, mixed in the query with another reply that mentioned the human element XD
Seems like with minimal oversight the human workers like to just say they did the requested task and make up an answer rather than actually do it (The task involved entering an address in Google maps, looking at the street view and confirming insofar as possible if a given business actually resided at the address in question, nothing complicated)
Edit: woops, mixed in the query with another reply that mentioned the human element XD