Mechanical turk is abusive. It bypasses the laws that were designed to protect the most vulnerable, and it will take some time for new laws to catch up that can regulate operations like mechanical turk. Just because new services and start ups have out maneuvered the law making system does not make those services ethical.
"Workers relying on such low wages and unstable employment are not likely to be able to educate their children enough to escape increasingly high rates of unemployment. A sustainable form of crowdsourcing will require forms of collective governance that mitigate the effects of market competition on those treated as mere links in a chain of algorithmic logic."
According to the nytimes article, 70% of the workers are women. For me, the possibility of paying an american woman 25 cents an hour so that I can get some press coverage is disgusting. When we needed data entry done, we hired some of our out of work friends for about $12-$15 per hour. It served our needs just fine and cost us about $1500 total.
For fast turn arounds, I see why you would want mechanical turk, but planning ahead can give you the possibility of treating people ethically. I don't agree with any systems that perpetuate poverty. Mechanical Turk obviously provides a very valuable service to businesses however. Does anyone know if there any systems like mechanical turk that guarantee at least minimum wage is being paid to the workers?
Love your passion social justice. I share it, and I can understand how a cursory understanding of mTurk may feel exploitative.
That said, it's clear neither the NYT nor the Daily Beast understand MTurk users - and because of that, they mistake a hobby for a job.
For many people in the US, MTurk isn't about making money, it's about doing something while you're bored. It's not a replacement for a job, it's a replacement for Farmville.
"Ever wonder what our labor market would look like without minimum wages or labor law protections?"
If Turkers were paid with a virtual currency, there would be no comparison to minimum wages - but because they're paid with cash, the puzzles/work exists in a weird world between work and play.
Also worth mentioning, I intentionally price my HITs low so as to target turkers who aren't looking to supplement their income. It only takes 5 minutes of trying to find one person's email address to realize $.03 isn't worth it - if your goal is to make money.
I may be still missing the boat, but if Turkers really are doing this as a hobby - is it still exploitative?
Hi Justin, that is a good point. According to this study in 2010 on page 4 http://sites.uci.edu/andrewzaldivar/files/2012/05/SocialCode... about 82% of the survey responders answered that the income was not essential, but the other 18% responded that the income was either important or essential to make ends meet.
Doing some turking to buy extra things while someone else pays your bills isn't exploitive in my opinion. But, for the 18% who are using is to make ends meet I do think it has a very real possibility to be exploitive. That's where regulation could help, in ensuring for those people who really need it have the safeguards in place to prevent the exploitation. I am far from an expert on Mechanical turk, and if it could guarantee that minimum wage was being met then I would not be against it.
(Disclaimer: I work for Amazon, but not on anything related to mTurk)
Is there reason to believe that people are using mTurk full-time as a significant source of income? I've Turked a bunch -- and know people who do it for a couple hours a day -- and my understanding is that its overwhelmingly used as a way to pick up a few extra bucks while catching up on TV, a slightly more civilized replacement for clipping coupons or filling out paid surveys.
According to the same 2010 study I mentioned in another comment, about 7% use it 30 hours per week plus. That's not a huge number but I imagine that those of the 7% would consider it full time work. http://sites.uci.edu/andrewzaldivar/files/2012/05/SocialCode...
I was doubtful at first, because of the need for both a computer (and Internet connection) in a part of the world where 25 cents an hour would be a decent lower-middle-class income.
Maybe there's a use for a "Fair Trade Mechanical Turk" - someone sets up an Internet work space for people in developing nations and FTMT sets minimum wages?
Here's where I get confused by this economic justice:
I have a little personal project and there's a piece of it that could be hired out on mechanical turk.
The project isn't worth much much money to me, it really is just a point of personal interest. And the little task is interesting enough that a few people might actually do it for free.
So I could ask for volunteers. Worse things have happened: how many times do people fill out surveys completely for free?
Surely no one would say I'm exploiting anyone by allowing them to volunteer their time? After all, it's their free choice if they want to waste a few minutes on my silly project.
But apparently I would be exploiting them if I offered a modest sum as a small "thank you".
Cognitive bias is human nature. The example I read is you'd get many more lawyers working pro bono for charity than you would offering a low wage. This of course extends to our perception of other people doing it.
I for one think we should remove minimum wage entirely but give everyone adequate food and healthcare. If people didn't go bankrupt after a serious injury and had enough food to wait out crappy offers, an appropriate wage would appear naturally.
It wouldn't be exploitative if the money was merely a thank you.
But the money wouldn't be merely a thank you. Not for everyone.
If you were making a standard offering on mechanical turk it's likely* that you would find workers that are in poverty, needing it as income, finding themselves working obscene numbers of hours. As example, a system that tries to get someone to work 90 hour weeks can easily be worse than no income at all.
Your attempt to invoke strong emotions may be a useful rhetoric instrument, but it is absolutely not adding anything to those doing the jobs at MT.
How in the world can you even waste your time on criticising use of a high-tech product that is available only to those with a computer and access to internet? Surely there are those who are exploited much much more than users of MT? Especially, if your own link shows that 82% of survey responders answered that the income was not essential.
This is game theory all over again. If "ethical" people wont use MT, only the bad guys will reap the benefits of the increased efficiency. Shaming people doesn't help much if the "right" option is detrimental to their incentives.
Moreover, your US-centrism is showing in full force. Don't you think that paying an Indian or Romanian 25 cents/hour is disgusting? Ok, you hired some of your friends for $12-15/h for mechanical job. So ethical. Why didn't you do that in an English-speaking part of the world where citizens live for less then $1 per day. Wouldnt it be more ethical?
If you are concerned about education of poor children: spread knowledge about moocs, help with youth coding camps, tutor people or do free seminars.
And if you are really-really concerned about the Mechanical Turk then by all means go ahead and teach people who are on MT how to earn more, that would be so much better than complaining on HN about the ethics of MT.
I am not trying to shame anyone, and yes, I am involved in researching solutions for a project that aims to help people in poverty empower themselves through providing data processing services. I also accept lower wages to be able to work on projects that are beneficial socially and have rejected offers to work in industries like high frequency trading that were very lucrative. I have spent hundreds of hours volunteering in my life teaching reading, ged, and esl in low income and even volunteered to teach ged material in a jail. We can all help in many ways, but the main point is having good regulation and rules is beneficial to capitalism. The best solution would be proper regulation of mturk to make sure that all businesses are on a level playing ground, whether they use mturk or another solution that at the moment may be more expensive in order to ensure minimum wage. But until that regulation happens, or mturk ensures minimum wage, I won't use mturk for any project.
Rob, as a software engineer at google, I would not expect you to to be able to relate to those without opportunities. In fact, there is some evidence that utilizing your brain to solve technical problems can restrict its ability to empathize. Of course amazon has no gun, but that does not mean that poverty in itself is not violent.
As you like science and data, here is a read for you:
http://www.sciencedaily.com/releases/2012/10/121030161416.ht...
Perhaps he's empathic enough to realize what it's like to have few choices, then have a well-meaning but paternalistic social reformer swoop in and take some of those choices away, pat you on the head, and drive off back to his nice neighborhood.
There's another way of thinking about it, that involves no paternalistic behavior. The government forces you to put an accurate price on transactions. For example, say you have a year-old good-condition car. As I understand it, you can sell it for around what it's worth, considering circumstances, or you can give it away, but you can't sell it for $50.
I don't have any good links or books unfortunately. There are a few people close to me that have a really great ability to empathize with others. They instantly understand social situations that will take me forever to get the dynamic. They respond to the needs of others much more quickly. I've tried to learn as much from them as possible - and often remember them when making decisions that have an impact others. But, I still have a long way to go.
It's just an intuition, but solving problems as a job might make you see every issue or bad situation as something to solve.
Not that it's wrong, but the focus slightly shifts from what the people are feeling to how the problem could or should be solved, and we might take less time or emotional resource to empathize. The situation itself can also be seen less as a tragedy than some sad transitory state waiting to be solved, and we'd be less sorry for someone if somewhere we think it might be just temporary.
I'd too would love to find studies or evidence about this subject.
The study shows that analytical thinking prevents/reduces empathy in the moments right after engaging in it (few seconds break). It doesn't, as far as I can see, prove anything about long term effects, and in fact, it specifically says that at rest, people naturally switch between the states.
It doesn't even sound good on paper. You don't have to work if somebody puts a gun to your head, either. You have the freedom of choice to take the bullet if that's your preference.
But isn't that real problem? Not that MTurtk is available, but that people have no better choice?
Attacking low paying jobs satisfies our moral outrage, but it doesn't actually help those wrote are left with nothing. We should concentrate in making sure people don't have to choose MTurk out of desperation, and banning it doesn't achieve that.
This doesn't mean Amazon and companies like it are innocent in this situation, of course, but it bothers me that people concentrate on the cosmetics instead of the systemic problems.
> Attacking low paying jobs satisfies our moral outrage, but it doesn't actually help those wrote are left with nothing.
Yes it does, the minimum wage goes hand in hand with the social safety net. Banning low wage jobs pushes those with no better choice into the safety net which is better than letting them try and work 90 hour weeks at jobs that don't pay a living wage. This frees up their time to actually do something about the problem rather than being stuck just trying to feed yourself while you work yourself to the bone.
So what you're saying is that they actually have a better option, but they're unable to realize that, so we should decide for them because we know best?
Or is the safety net not available to them because MTurk exists? Because if it's the latter, I don't see why wouldn't you just make it available, MTurk or not.
I'm not talking about MTurk specifically, but about all below minimum wage jobs.
> but they're unable to realize that, so we should decide for them because we know best
They are we, we are they, we're one society and if we democratically set a wage floor, then yes, we know best. That's rather the point of democracy, to remove choice about some things from the individual and make them together because it's better for us all despite what any one individual might choose.
> Or is the safety net not available to them because MTurk exists? Because if it's the latter, I don't see why wouldn't you just make it available, MTurk or not.
You'd be presuming the rules behind social programs are logical, they are not, they are designed by committee and as such can be haphazard and often illogical. Making money can actually hurt you if you're on the safety net because there's a hole where you're job would disqualify you for benefits that could exceed what the job would pay. Of course that varies state by state so there's no single answer to the question.
So until the safety net is more logical and lets you work without putting you in a worse position, then it's rational to simply not allow jobs that don't pay a living wage. Many have proposed a basic income to avoid this problem.
That's rather the point of democracy, to remove choice about some things from the individual and make them together because it's better for us all despite what any one individual might choose.
Which is exactly why the phrase "tyranny of the majority" exists, and why "democracy" is just a euphemism for "mob rule" at the end of the day.
He also said, "I propose that 100,000 degenerate Britons should be forcibly sterilized and others put in labour camps to halt the decline of the British race."
And this, "I do not understand this squeamishness about the use of gas. We have definitely adopted the position at the Peace Conference of arguing in favour of the retention of gas as a permanent method of warfare. It is sheer affectation to lacerate a man with the poisonous fragment of a bursting shell and to boggle at making his eyes water by means of lachrymatory gas. I am strongly in favour of using poisoned gas against uncivilised tribes. The moral effect should be so good that the loss of life should be reduced to a minimum. It is not necessary to use only the most deadly gases: gases can be used which cause great inconvenience and would spread a lively terror and yet would leave no serious permanent effects on most of those affected."
And just in case you couldn't tell, he confirms his inner monster: "I think a curse should rest on me — because I love this war. I know it's smashing and shattering the lives of thousands every moment — and yet — I can't help it — I enjoy every second of it."
So what, am I supposed to look up to this man and accept that Mob Rule is the best way to govern just because he says so?
But whatever man, people will just keep believing what they're believing. But at the end of the day, those who would use violence to enact their own ends will continue doing so.
You'd rather the minority rule? You have some fantastic idea for government that doesn't involve pissing off anyone? You whine about mob rule and democracy as if it's evil or something when it is in fact the best form of government we've figured out yet, despite it's flaws. What the hell is wrong with you? Are you some wackjob hiding in a bunker somewhere plotting the downfall of the government?
Yes, so it is relevant to note that democracy does not prove rightness or wrongness of an idea. As many others have pointed out, democracy is the best form of government because of two reasons :
1. It ensures that absolute idiots cannot come to power
2. It reduces conflict ( 'You don't like what the government is doing? Too bad, you voted for it!' ).
You'd be presuming the rules behind social programs are logical, they are not, they are designed by committee and as such can be haphazard and often illogical.
The same can be said about labour laws, including minimum wage; by that argument, any discussion on policy change is irrelevant.
And some social programs do paint a better picture, such as the EITC[1].
Besides, my argument is more general than whatever US social programs happen to be in place, since I'm not even from the US or living there.
So until the safety net is more logical and lets you work without putting you in a worse position, then it's rational to simply not allow jobs that don't pay a living wage.
And maybe I would agree with you if I was a labour legislator but had no power on safety net policies. But since my (and, I assume, your) power to change any of them is the same, I don't see how is that relevant.
Care to elaborate what is the exact mechanism by which MT perpetuates poverty? Given two worlds, one with MT and one without, what exactly are the opportunities that poor people are deprived of in the former which are available in the latter?
Basically, by spending so much time doing work for peanuts to make ends meet or as a primary source of income, they become reliant on that income and will not find opportunities elsewhere. The argument is that mTurk perpetuates and thrives on cheap labour.
The solution is obvious, make the decision for them and deprive them of that source of income or legislate minimum wage rates... This in my opinion would be very shortsighted and harmful.
Certainly they performed jobs for 25 cents an hour, but that rate just isn't sustainable if you actually want to get fast, reliant results.
Just as the requesters on MTurk are smart, so are the workers (aka Turkers). If you hang around the market place long enough you will quickly discover that your time is worth much more than 25 cents an hour.
The Turkers are smart; high throughput is the only way they are going to make any money.
As a requester, someone posting work, if you don't respect that in terms of 1) streaming-lining your instructions to simplify the work for the Turkers, and 2) giving a descent wage relative to the work needed
The Turkers will go elsewhere and your results are just going to trickle in.
MTurk is actually a perfect marketplace, as a worker you can see ALL of the jobs that you are qualified, all of the competing bids for a piece of your time in one consolidated place, and you can choose how and when to work instantly or whether you want to work at all.
25 cents per hour is roughly 45 $ a month. (8 h/day, 22 working days/month) I guess someone who would work for that wage lives in a rough country. As we all know from the news and documentaries, those countries often have horrible working conditions, like 60 hours or more a week and physically hard work. Isn't a mTurk job better than that? Also don't forget the lower prices in such regions.
The thing is this: NGOs and friends are trying since decades to solve humanitarian problems. Unfortunately they suck incredibly at what they are doing. The countries stay poor, the working conditions stay bad, war and crime continue.
I strongly believe that economical prosperity automatically leads to better living and working conditions. Just look at the African countries the Chinese got their hands in.
BTW, don't forget that there is not only competition between the workers but also between those submitting jobs. I.e. many mTurk users => more competition on the income side. Even better: competition with same working standards. You don't even have that in the U.S.!
The comment you're replying to very clearly focused on such a wage as being bad in the US. Sure, if the worker is somewhere that 25 cents an hour is livable, then everything works out okay.
As one data point, I spent a while during college and took Turking seriously as a source of income and averaged about $7-$8 an hour over that time period. Not awesome, but not bad either for an always available, willing, form of employment.
"Always available" and and incredibly flexible form of employment. There aren't many jobs where you can sit down, make some money, when you need to take a break (kids screaming, etc), you can stop, resume again when you're ready. The flexibility is worth something.
$0.03 per email seems awfully low, and if you do 1 article per minute, you're making $2 an hour, which is definitely not cool.
But how long does it actually take to find one email from an article? Maybe 10-15 seconds? Then you're talking about $7-$10 per hour, which for unskilled work doesn't feel especially exploitative.
The real draw of systems like mechanical turk is not that labor is cheaper (it usually isn't), but that's it's more flexible, i.e. you don't have to hire and train people for any particular task.
I don't disagree with this sentiment, but I don't think it is correct to say that Mechanical Turk itself is abusive. Mechanical Turk certainly CAN be abused, but it also CAN be used ethically. For example, we regularly use MT to complete tasks that pay workers effective hourly wages in the range of $8-20 per hour, and I know several other requesters who pay similarly.
It is probably most accurate to say that the lack of worker safeguards on MT attracts many users who pay rates that are (IMHO correctly) classified as abusive. Part of the issue with MT is that traditional worker protections like a minimum wage no longer make sense when you have compensation that is paid on a per-task rather than per-hour basis. Under a per-task compensatory system, faster workers will always earn more money (on an hourly basis) than slower workers. For the tasks we do, it is not uncommon for subsets of workers to earn effective hourly rates almost three times as high as others. In many ways this is good - incenting workers to spend their time on tasks that they can complete most effeciently (and away from tasks that they are less suited for) helps promote the most efficient allocation of labor resources.
But how do you ensure that all workers will always be treated fairly when a minimum wage no longer makes sense? What do you replace it with? Its not clear that there is a simple replacement that works in the micro-tasking context.
Our platform (http://houdiniapp.com) helps businesses complete projects on MTurk. One of the things we realized is that many requesters pay low wages simply because they are really bad at estimating how long tasks actually take to complete. We’ve been able to solve this (somewhat) by providing tools that provide real-time feedback on average completion times in order to provide direct visibility into real hourly wages. However we allow users to set their own compensation and some users still end up paying effective wages in the $2-3 (which we do not encourage, but don't have any simple way to prevent).
I thought most folks (at least US based) performed the work on the side. Such as receptionists with lots of down time or late night security guards. A lot of this is just busy work. Click this if it's porn... fill in an email from the linked page... categorize this image. Am I totally off base?
The substantiation of the $0.25/hour claim is that they were able to find three people who were willing to work for that much on one task, once. I think this is extremely weak evidence to base a lot of outrage on or to make a claim of what the market hourly rate is on MT.
Are the small number of people depending on Mechanical Turk for real money actually capable of doing regular minimum wage labor? We already have broad exceptions to minimum wage laws that allow the marginally employable (seriously disabled, mostly) to work for less than customary sums of money while also collecting disability benefits.
Having someone who's medically unable to work or who is primarily a full-time caretaker have a home job that makes them a supplemental income on the side, even at $.25/hr, doesn't sound like an unethical or abusive system to me.
(also, turkers who are doing transcription a lot probably know how to do 4x pitch corrected fast forward on their audio player and have a good enough ear for voice to pick out the odd words)
Paying fair wages means better work output, which means you can get higher quality results, and actually means lower total cost. It's better business than trying to scrape usable results out of bottom-dollar markets like Turk, and far easier, to boot.
Check out what Peter Schiff says about the minimum wage. He has some good, reasonable arguments IMO. The minimum wage ultimately hurts everyone. Minimum wage = fewer jobs (quantity and types of jobs) and higher prices.
I've mentioned before that I would pay to use this product.
If you had the same tool that worked with a blog search API not only would I pay for it, but you could sell it to SEOs all day long. Just need to find a decent blog search API; I haven't had much luck.
If someone here were to market this product, do you have any suggestions as to how they could reach other customers as well, enough to make it a sustainable business?
It is a good idea for a service, if possible please consider something like https://www.mobileworks.com/ as vincvinc mentioned below that tries to be socially responsible for crowd sourcing instead of mturk.
At the kind of scale I would use it at I would just hire personal assistants.
That said, I've actually used mechanical turk to earn money in some of my low points (was living in my car working on a startup, out of cash, crazy/long story). I didn't make US minimum wage, but if I were living in Bangladesh, for example, it would have been OK money.
It could be any range of blogs; it's one of the most effective and least-used means of marketing.
Say I'm a bowtie company that sells to designers; I could look for all the fashion blogs by searching "bowtie" or "necktie," etc. Just generic blog outreach, whether it's little blogs or mommy blogs, would be very useful. Not only do you get some traffic from those blogs, but you get the link juice of some really good links.
Good question. Overall quality was high (you're able to see the results as they come in and if they look funky, you can reject them and you won't have to pay).
The masters qualification is handed out to the top tier workers. It's for people who basically use mturk for a full time job and 'usually' output extremely high quality work.
The masters qualification is supposed to open up the higher paying tasks. They will calculate the cost and if they aren't earning, at minimum, .10c per min or better, they will abandon or return your hits. Keep in mind that probably less than 1% of all turkers have that qualification. It's extremely tough to get.
I mean no offense, but requiring a masters qualification for your (being blunt) junk hits is like requiring a PhD to flip burgers at McDonald's.
You might find this service I built useful: It queries the Bing API for different combos of the name/domain (John.Smith@nytimes.com, JSmith@nytimes.com, etc) and returns their counts, and then deeplinks into a Google verbatim search of that email address.
dude.... screw that KickStarter thing ;-)
Build this as a business! I've literally been trying to solve this problem for a month. Freelancer.com sucks and you can't buy cheap targeted email lists. You saved me a huge headache. Thanks!
For this particular example I don't understand it.
But I look through reddit's "What is this thing?" subreddits to try to find what the thing is. I do that to improve my skills with search engines. (Really, try it sometime. A blurry photograph of a rusty lump of metal, with no context other than "I found this in a field" makes you think hard how to find things).
3 cents is no where near enough to make me want to do this more, or to persuade me that I'm not wasting time online, but it goes in the free money jar.
I guess there's a risk that offering money puts off more people than it attracts - more people will do something for free than if you offer a tiny payment.
Productize this! I'll pay you today for an easy to use service.
I am absolutely going to give this a shot for our next product release. I love that it simplifies the connection to relevant journalists. It's truly a win-win.
That's pretty much the process I used to launch some apps -- except that I like to get the contacts myself and read some other articles from the person, so I can add a personal touch to the emails (it will increase your chances with the top journalists).
Definitely should be it's own product. Only snag I found was for double byte characters. Can anyone give me a hand with getting the titles and desc's to pull in properly if the language is in double bytes?
It used too, but they've had a lot of problems. Apparently there were a lot of scammers from India so they (temporarily?) ceased allowing new accounts from outside the US.
"Workers relying on such low wages and unstable employment are not likely to be able to educate their children enough to escape increasingly high rates of unemployment. A sustainable form of crowdsourcing will require forms of collective governance that mitigate the effects of market competition on those treated as mere links in a chain of algorithmic logic."
http://economix.blogs.nytimes.com/2013/03/18/the-unregulated...
Workers will do jobs for as little as 25 cents an hour. http://www.thedailybeast.com/articles/2011/06/22/americans-w...
According to the nytimes article, 70% of the workers are women. For me, the possibility of paying an american woman 25 cents an hour so that I can get some press coverage is disgusting. When we needed data entry done, we hired some of our out of work friends for about $12-$15 per hour. It served our needs just fine and cost us about $1500 total.
For fast turn arounds, I see why you would want mechanical turk, but planning ahead can give you the possibility of treating people ethically. I don't agree with any systems that perpetuate poverty. Mechanical Turk obviously provides a very valuable service to businesses however. Does anyone know if there any systems like mechanical turk that guarantee at least minimum wage is being paid to the workers?