While there maybe subtleties in "public access by humans" over "automated scraping" - these belong in the "business dispute" side of the fence. Using the CFAA is by definition - playing hardball - it includes Criminal Penalties in some cases.
What really amuses me is that in the Tweet highlighted, Craig is indeed implying that it is his money going to the EFF - who Opposed him in the legal case that lead to the money. So he should be called out for egregious bullying - using the threat of Criminal Sanction to get his way.
If an HN-friendly business (whatever that is) pulled the same move it would also be equally worthy of opprobrium.
I think what you're trying to imply here is approximately as accurate as Techdirt's claim that Newmark "made the CFAA worse".
The CFAA isn't the only set of laws that has both a criminal statute and a civil cause of action. Newmark didn't --- can't, in fact --- threaten anyone criminally. Craigslist's recourse to a statute that included a criminal component added no rigidity to the ball they were playing with, whatsoever. Since they were suing to enforce their own terms of service, the CFAA was the correct statute to cite.
Other people on this thread have also suggested that recourse to the CFAA was somehow a nuclear option for Craigslist. It was not. People think this because Techdirt is misleading them, in order to whip up rageviews. You should be irritated at them for doing that (it's pretty much their whole M.O.).
And it works :-( judging from the slacktivist chatter among my less intelligent (previously determined, not judging based on politics) but still technologically geeky FB friends.
Craigslist continues to destroy its competition by maintaining its walled garden. This wouldn't bug me as much if CL was actually serious about improving its user experience, which has more or less been the same for the past decade (just bad enough to be usable).
I don't mind it being spartan - but that aside there are usability issues. One simple example is the reason things like PadMapper and HousingMaps became popular. Another example is that their categories are often hard to filter through. Lastly, there are trust issues - something like "connect to Facebook" would really go a long way in some transactions.
If anyone ever needed proof that monopolies crush innovation, this would be a good starting point.
I don't see the connection between bullying being acceptable and improving a site's UX. If it's wrong then it's wrong regardless of whether or not they steal competitor's ideas when they crush them.
CL is popular because it's simple, easy, cheap. They have tons of competitors that they manage to beat, so they're doing something right. Companies that want to horn in on them should create the marketplace as well as the front end, but they would fail at it so they're just left with complaining.
Or, they "dumped" free classified listings into the market, killing off competitors (newspapers), giving CL a monopoly.
Of course they "manage to beat" competitors, that's not hard for a monopolist. But it doesn't equate to "doing something right."
The whole suit is ridiculous from the consumer/society perspective. You can't argue that scraping Google to get CL listings harms CL in any way whatsoever. So the only TRUE reason to sue 3Taps (or their customers) is to retain the monopoly, plain and simple.
I'm starting to think that, on balance, CL may no longer be a net benefit to society.
Even if CL's interpretation of the CFAA is legally correct, this is an abuse of it by any measure. Suing people for accessing your publicly available website is absurd. The fact that Craig would publicly claim the money was a donation, after he obtained it by exploiting a very dangerous view of the CFAA and bankrupted a company that didn't do anything wrong, is disgusting to me. I've always found Craig Newmark to be a little creepy; now I think he's squarely in scumbag territory.
Why is it absurd to first withdraw permission to publicly accessible service and then sue if the individual continues unauthorized?
I can easily imagine a public irc channel that ip blocks abusing users that post goatse links (not illegal in itself). If some of those users than bypass that block, is it unreasonable that the owner would want to sue in order to stop the abuse from the then unauthorized access to the irc server?
To me, I think the difference is that the guy posting goatse links is causing actual harm, while the guy scraping the website may have basically no impact. Only cases where there are actually meaningful damages should end up in court.
The company scraping Craigslist is not having "no impact". Craigslist didn't sue out of pique. They sued because their site was being forked, and they are not an open source project.
It seems reasonable to disagree about whether companies should have the right to prevent forks based on their public data sets ("can you fork the phone book?"). It does not seem reasonable to suggest that Craigslist sued simply in order to be tyrannical.
The phone book is an excellent example, because rather famously[1] at one point a powerful company tried to use IP laws to prevent someone from forking the actual phone book... and the US Supreme Court ruled that it WAS permitted. That precedent does not apply to this case which is based on different laws preventing access to the data not preventing contacting it.
> They sued because their site was being forked, and they are not an open source project.
Or, for that matter, because somebody was using up their bandwidth, which is a finite resource, for something other than the intended purpose of the site.
I'm talking specifically about the act of scraping, not about what is done with the data afterwards. The post I was responding to referred to a general scenario, rather than this specific lawsuit.
To focus on the act of scraping itself, consider a case where they download the data and study it privately. That should not go to court, even if they broke the ToS, unless there are measurable damages.
And because of the nature of 1030g, it couldn't go to court without measurable damages, because CFAA civil claims can only pursue economic damages. So, in a real sense, the implication of using the CFAA in this case is the opposite of what Techdirt manipulatively implies that it is. CFAA limited the scope of this case and reduced the stakes.
edited: this said "for 3taps" before, but I'm not sure that's accurate.
It's fair to point out they also sued after they issued a warning shot, in the form of a C & D.
I like your analogy of the site being forked. I don't believe for a 2nd that the companies scraping the CL data and putting a pretty UX on it were doing it out of any sense of altruism.
It's reasonable that the owner can sue. But you don't need CFAA to sue someone and it opens the door to serious criminal penalties which, no I do not think are reasonable for an IRC troll or for a Craigslist mapping tool.
You're the second person in this thread to suggest that the CFAA was somehow a nuclear option.
You're saying this because Techdirt is manipulating you.
The CFAA is in fact the logical private cause of action for suits to enforce terms of service. It is literally the section of the US code about enforcing ToS's.
A civil suit is a civil suit. CFAA-derived civil suits are not especially more scary than other civil suits. It's scary to be sued by Craigslist no matter what. The CFAA has nothing to do with it.
You should be irritated at Techdirt for misinforming you solely to generate rageviews.
CFAA is necessary so people can defend their property from invaders.
You'd file criminal charges against someone who breaks into your home every night to hurl racial slurs at you and keeps picking your lock no matter how many times you change the locks; why wouldn't you file criminal charges against someone who comes on to your channel/forum/etc. every day to harass your users and uses a number of proxies to change their IP no matter how many times you IP-ban them?
Imagine if someone had sued Google for basically doing the same thing back in 1998. Would it exist today? Would we have this great quality search engine that has made our lives so much easier?
Google in fact does honor requests not to scrape websites, and honors them at Internet scale. Padmapper could, meanwhile, reasonably be defined by their unwillingness to stop scraping a single site whose lawyers made it clear they did not want to be scraped. So I'm not sure your comparison is valid.
Maybe we'd still be using something opt-in like Yahoo Directory, and maybe the web would be better for it. Yahoo directory was slower but in many ways higher-quality; it would've kept the web more decentralized (e.g. Wikipedia would not be the top result for half of queries). Of course there would have been short-term costs to that route.
"decentralized" doesn't magically mean better! I personally believe wikipedia is one of humanity's greatest accomplishments. It's been an enormous group effort.
I have a copy in my phone, unbelievable amounts of knowledge in my pocket, several libraries full. Would it somehow be better if information were harder to find?
I think downranking Wikipedia would make some information easier to find. Often a subject-matter-specific site has a better page than Wikipedia on a given topic - but Wikipedia has more google-juice, so people are directed there instead.
There is certainly value in presenting everything via a consistent interface, as Wikipedia does. But there's also value is subject-specific interfaces. And I find Wikipedia's notability criteria very problematic; Wikipedia can't be treated as a sole source of truth when it simply refuses to include any information about certain subjects. (E.g. I wanted to know about a band I'd seen once, so I looked them up on Wikipedia only to find no page; I wondered if I'd spelled them wrong or some such. Then I looked them up on TV Tropes and found a useful page about them.)
Since everyone here liked the Padmapper UI over Craigslist they were OK with what they were doing, but if this was happening to an HN friendly business the story would be entirely different and there would a massive outcry. The hypocrisy is at least to me highly entertaining.
Very! Google provides a way for you to opt out. This is not the case of "Oh, I went to their website and they sued us out of existence." This is more of a case of they were asked to not do something and they said fuck you I'll do it anyway. I don't like CFAA but I'd say these 3tap people failed at being nice to others.
Yes, I remember those discussions here in HN. The simple fact was they had built and entire product around someone else's data. They maintained an "f-you" attitude even after the cease and desists. Those companies got hammered because of their arrogance. They could have built a CL-killer but instead they built a CL-piggybacker but did so in a way that violated CL's terms of use.
People on HN often complain about YC backed startups if they do morally questionable stuff (see for example any thread about that installer bundling startup whose name I can't recall right now). The HN community isn't as bad as you make it seem.
Dont forget the hubub over YC investing in that adware bundler company (InstallMonetizer), couldnt stop hearing about that for weeks[0].
I certainly think there is a trend towards being positive towards YC properties on HN (surprise!) but if they do something bad they will be excoriated just like the rest.
Yes. And there are tons of other startups and probably also YC companies involved in heavy scraping, aggregating and selling that information to other folks.
So sue on that basis, not on accessing the website.
I think most of us are not so unhappy about the suing, although we may disagree with it. We're very unhappy about using CFAA, which passes the line into criminal penalties. So we're talking about initiating a process against someone who scraped your site that could land them in jail.
You, too, are repeating the false trope that the CFAA is an egregious overreaction by Craigslist. Not only is it not that (it is literally the federal private cause of action specifically intended for enforcing terms of service), but it's more limited than other civil causes of action: plaintiffs can sue only for certain types of damages.
It's also preposterous to suggest that Craigslist somehow "passed the line into criminal penalties". Just because a section of the law also has criminal offenses specified does not make all invocations of that law criminal. Private parties can't criminally prosecute people. There was, obviously, no risk of criminal charges in this case.
Techdirt is, as usually, being manipulative in how they frame this story. The are the Internet's foremost amplifier of online law misinformation.
He's not "creepy". That word has just become too cute? Its been so overused by gossipy people? I not implying you're gossipy--I'm just tired of overused words that come into fashion: "Really?", "Dude","tool", "noob", "tldr", "Hipster", and every expression that turns into a cliché?
I've never met the man, but he seems to respect money? He doesn't look at the Internet as a way to become filthy rich?
He seems to despise the filthy rich? I like that he takes the bus. The world needs more guys like this!
As to the lawsuit, I think he looked beyond simple data scraping; he knew a lot of us out here say one thing, but are looking to capitalize, and live like Lord Fauntleroy in the end? One day, hoping to give that TED talk, and let's not forget the finally marrying the cool chick with all that slick loot? And getting the eventual divorce?
Craig said no to the real money! I really thought he would sell out by now, but has stayed pretty modest.
Can't we just ban Techdirt submissions for their terrible and obviously biased reporting? Everything from them is blatantly one sided and not even trying to actually represent the truth.
From 3Taps own website right now:
"As part of the settlement, 3taps and its founder, Greg Kidd, have agreed to pay craigslist $1 million, all of which must then be paid by craigslist to the EFF, which supported 3taps' position on the CFAA in this litigation, and continues to do great work for Internet freedom"
Techdirt is in a category of sites whose submissions are penalized by default on HN. This category is for sources that produce a lot of fluff but also the occasionally substantive piece. There are various ways for the penalty to be lifted, one of which is moderator review, which is how it got lifted here.
Your description might be fair in general, but this article is more substantive than the typical riler-upper. If you know of something that it misrepresents, by all means correct the record. But it doesn't misrepresent the info you quoted—on the contrary, it communicates it clearly and includes the exact same quote.
2nd graf: suggests Craigslist "abused" the CFAA to "kill" a company. Instead, Craigslist relied on the one private cause of action in the US code specific to enforcement of ToS's --- a limited cause, with limited damages. It was inaccurate to call this an "abuse" (it's the whole intent of 1030g!) and misleading to invoke the CFAA boogeyman over it.
4th graf: Claims Craigslist relied on a "tortured" definition of CFAA, linking to another Techdirt story about a judge's upholding of Craigslist's claims on two different counts. "Torturous" here means "the suggestion that access was unauthorized after Padmapper received a cease and desist from Craigslist's lawyers, and changed IP addresses to evade a ban.
6th graf: Techdirt grossly misrepresents Orin Kerr, perhaps assuming readers won't click through to the Volokh story they quote out of context. Kerr sees the ruling as a missed opportunity to more rigidly define "unauthorized access" as "circumvention of technical controls" but does not disagree with the ruling. HN readers in general would not be happier in a world where Kerr's view of the CFAA was reliably enforced.
Most importantly, I think: the article also never makes the argument it teases in its headline: at no point is it ever made clear how Newmark has "made the CFAA worse".
This is a defect density rate worthy of early-period PHP Wordpress.
Your normal instinct about Techdirt is almost always going to be right. It's not a good site. I'm glad it's penalized. Penalize it more.
Hey Thomas, big fan of your comments generally, but you're mischaracterizing what we did. We never issued another server request to them again after the C&D, let alone changed IP addresses to evade a ban afterwards. This is uncontested, and we weren't included in the CFAA claims. We were fighting for the right to summarize listings and link to them, not for the right to access their servers when they don't want us to, nor to include the full text of the postings on our site.
Sorry, I was working from the link Techdirt provided to back up the "tortured" application of CFAA. I really wasn't trying to characterize exactly what your team was doing. But I was also imprecise and I sometimes forget that the principals of these stories are on HN, too. I apologize.
I found the article to be misleading and of pretty poor journalistic quality. It reads like a rant and I would hope that HN would not make an exception for a Techdirt rant.
For example: the line, "Craig himself has contributed to this misleading perception with this tweet implying he's giving his own money to EFF" was especially confusing since the proceeds of a lawsuit are in fact "his own money".
Fully agreed about rants in general. But I also agree with the commenters pointing out that complying with a lawsuit settlement is really not the same thing as a donation. That, plus that the story is significant, is why we haven't penalized this article the way we normally would. I'd be happy to swap it out for a more substantive, less ranty article, if one exists.
The idea that "unbiased" reporting exists is a modern myth. There is bias in simply deciding the topic to report on.
The modern idea that "unbiased" is somehow a virtue just leads to the CNN phenomenon where every story has to have two sides regardless of what the situation actually is.
One of the purposes of journalism is to make the decisions about what is "news" and what is junk. Obviously these criteria are going to depend on a lot of personal interpretation, so the key to good reporting is to know and understand how the reporter judges a story - their bias - so their reporting can be properly weighted.
In this sense, techdirt is quite open an obvious with their bias, and is fairly consistent in style, which is something I wish I saw a lot more at "news" websites.
(how you interpret that bias is another matter entirely)
As for their content, while I agree techdirt can be one sided at times, I have never seen any serious lies, either direct or by omission. Maybe I've just missed them?
Speaking of which, I don't see how reposting a quote that was included in the techdirt article is supposed to be evidence of them being "one sided" or "not even trying to actually represent the truth".
That doesn't sound like a "donation", though? At the very least Craiglist tried to portray it as something they're doing out of the goodness of their hearts.
I'm sure Craigslist could have negotiated for a settlement that would have involved money (perhaps less money) going directly to them. And if Craigslist had done nothing the EFF wouldn't be getting any money. So they deserve at least some of the credit.
Giving away a $1M settlement is clearly a donation, and it's "out of the goodness of their hearts" when compared to the obvious alternative: keeping the money.
I wouldn't consider the result of legal settlement to be a "donation". A donation is a voluntary transfer of money. Had craigslist received $1M from a settlement and then voluntarily sent EFF $1M without being specified to in the legal settlement, then that would be a donation. The correct interpretation is that the $1M transfer is part of a legal settlement to resolve a lawsuit. I haven't followed this lawsuit too closely, but it sounds like craigslist might not have had a "open-and-closed" case, hence the need to offer these amenable terms to sweeten the terms of surrender, and thus hasten the resolution.
"As part of the settlement, 3taps and its founder, Greg Kidd, have agreed to pay craigslist $1 million, all of which must then be paid by craigslist to the EFF, "
Yes; specifically, if the proposed settlement was "3taps will give Craigslist $1 million, which Craigslist will have no restrictions on", then 3taps might not have agreed to it.
Maybe CL initiated the requirement that the money was going to go to the EFF after being paid to them? Seems like most of the ill-will related speculation might be just that until there is any evidence either way...
Craigslist is a great example of how network effects can be leveraged to form the basis for a sustainable competitive advantage. The product is an embarrassment and yet it has users simply because it has users. Newmark knows this and that's why he sues anyone who sniffs in the general direction of his data. I think he's a self-serving fink.
> The product is an embarrassment and yet it has users simply because it has users
The product is bare bones, but it works just as well as it ever has, which is a heck of a lot better than many web applications from the last 10 years, loaded down with animations and piles of tracking javascript and auto-playing video ads and broken reimplementations of standard browser features.
Personally I find it works better for me than most google web properties since about 2008 (e.g. the new versions of gmail and google maps), better than recent versions of facebook, better than yelp, better than a lot of current newspaper websites, etc. There’s something to be said for plain html websites.
I do wish craigslist would let other people scrape their pages and provide alternative user interfaces though. Pad mapper was a great help when I needed to look for an apartment several years ago.
I'll grant you that the basic functionality works. It ought to given that they've barely changed it in like 15 years. It's fine if they don't want to change it, but they've stifled -- through legal channels -- many frustrated attempts to enhance the product -- like cease and desists to dude's doing simple mapping visualizations for example. I think they're bullies with their position and it has served no one but craigslist. The network effects they wield create monopoly-like conditions and they use the court's to retain them.
[edit for grammar which alas is not quite perfect yet]
When I hear people say that Craigslist hasn't changed in 15 years, I have to wonder how much they actually use Craigslist.
The functionality has changed (improved) dramatically since 15 years ago. For example there is now a map view for many search results. Searches of "for sale" listings now default to a gallery view. In fact the entire infrastructure for accepting and displaying images is new; Craigslist launched as a text-only website, and embedding images was a messy hack for years.
In general the data structure has been enhanced to collect more and more finely grained pieces of information by which searchers can filter their results. To name on example I've used, selling a boat on Craigslist now has its own set of dedicated fields for attributes like length, manufacturer, model, power source, even the number of hours on the engine.
The one thing that hasn't changed is the visual aesthetic. It's still plain-Jane HTML. But there are plenty of ugly-but-functional websites that do very well: Wikipedia, Google, and even Facebook are boring designs on top of really rich server-side content.
Interesting. The product belongs to CN. If he doesn't want to "enhance" it, what gives anyone else the right to think they can step in and do it? If Toyota didn't offer a red Camry, do you think someone should be able to set up their own robot at Toyota to paint certain Camrys red?
If people are not satisfied with CL, they should build a better CL from scratch and stay away from CN's product.
You basically made my same point. I in no way said that people should not be able to paint red Camrys. I said they shouldn't be able to do it in Toyota's factory (by installing their own robot).
It doesn't follow the latest web9.0 flat^Wmaterial^Wlook-a-monkey! design trends?
The interface remains usably utilitarian?
They don't try to capitalize to create an uber-exit that everyone can write a congratulatory comment on YC/TechCrunch about? Great hustle Craig!
Or is it just that you're of the Uber-style ethical camp that believes in disruption-through-cost-externalization? In this case, actively extracting value from Craigslist, despite their protests, and using it to undermine their business.
For the record, I'm a fan of the retro design -- or I guess it's original in their case. In any case, I don't mind that.
It's that the product could be better -- as in better features -- in a bout a million ways. How about notifying me when something becomes available for instance... Or showing a map view of apartment listings or heck all listings... Or a bit a zillion search possibilities...like -- here's one -- searching across a whole region or farther for a given item... Or some kind of attempt at reputation building as an option...
That list took maybe 90 seconds... My theory is that they are lazy because they can afford to be and they aggressively defend their laziness by suing anyone who tries to make it better. Finks.
"I'm a fan of the retro design...It's that the product could be better"
Agreed. I gave up on my first experience trying to rent an apartment after getting obviously sketchy responses emails from my inquiries, that was after I had spent hours sifting through the listings and only messaging the few listing that didn't look like scams. It's amazing how difficult it was to find where the approximate location, which definitely would have helped with some very simple non-js map. It shouldn't be that difficult to get a valid description of what I'm looking for.
Ok it looks like after suing that poor dude, they added a map to the apartment listings finally. I stand corrected on that one. It was missing forever.
I still don't see anywhere to search across regions. If you've ever tried to find a hard to find car you'd understand what I mean there. I'd like to search in, say, all of California. Can't do it.
My point is that there's a lot that could be done that will not be done because they won't let people scrape even if they link back. So, for example, say I wanted to add a state-wide search. Even if all my search results pointed back to CL, CL would cease and desist me. This goes for other enhanced functionality as well. This is all legal, but see it for what it is: an aggressive use of their position to retain their position, which includes squelching any and all uses of their data. It's an anit-competitive move that let's them maintain what, in my opinion, is a very poor quality product for all the traffic and data they get.
At least in the UK version, they do have some kind of cross-region search now in the sidebar. (Which is pretty much essential here, because their region boundaries don't make much sense and people don't always agree on what region they're in.)
"Nearby listings" doesn't work like I'd like it to. It'd be great to be able to choose from multiple craigslist areas and run searches on them, distinct of their proximity.
They have made lots of functional changes in the past two years. Better image presentation; more sorts and filters specific to the category browsed. But before 2013 or so, it really was a POS 2000-era site. craiglook and padmapper, and others that put useful UIs over the dataset, did a real service to the platform and Craigslist destroys them.
If you're an entrepreneur, I would respectfully suggest you worry a little bit about the thought pattern that would make you say that Craigslist's "product" is an "embarrassment" and that it "has users simply because it has users".
Craigslist's product is its userbase. It is by all accounts a fabulously successful product.
You want be able to see past the AngularJS and D3 and iOS-first and responsive UX stuff to the meat of what a real value proposition is. You don't want to be held hostage to fashion.
Cynical part of me sees all the people saying "EFF should give it back to show integrity" and wondering if that wasn't the hope all along from craigslist.
I would hope that EFF does not give it back but uses it to continue the good fight or donate it to another, unrelated, charity if they believe the money is tainted.
Everyone is just frustrated that Craigslist charges so little for their services and hasn't tried to monetize it in some abuse way like a typical capitalistic company would.
Among my objectives is to eliminate the corrupting influence of money from technical hiring.
In the long run I want to cover every occupation in which one can find a job by applying through the employer's website.
I won't monetize it in any way. Or perhaps I could say I will monetize it by offering my consulting services through my website, as I always have. Providing free content on my site results in sales leads.
What I do regard as corruption, is when a recruiter convinces a manager that they can find qualified staff, then expect thirty grand in commissions by placing someone whose rigor mortis is not quite started,
I am good at my work, I take pride in it but recruiters commonly solicit me for perm or contract work for which I am completely unqualified. For example I do osx device drivers and windows gui so on a damn near daily basis I am approached by recruiters who want to submit me for windows device driver work, even at microsoft.
Their applicant tracking systems other do not support exact phrase matching or the recruiters dont know what it is. Neither do they ever attempt to read my resume, not even when they do submit me.
Many of my "colleagues" were obvious imposters who found work through recruiters. The recruiters dont care, they get paid, see.
sigh This isn't a problem with recruiters [0]. This is a hiring problem. Hiring isn't very easy, and it seems that a huge number of companies don't know how to hire.
You have opinions and your HN profile indicates that you've been programming for a long time, but that doesn't appear to have any relevance to your opinions on hiring.
Oh, BTW, you have some mojibake in the Education section of your online resume.
[0] Numerically, most recruiters suck. This has nothing to do with hiring. :)
Oddly that page has utf-8 encoding. The mojibake were em-dashes in the original OO document, I don't clearly recall but likely copied it to the clipboard, then to textwrangler for OS X than marked up the HTML.
textwrangler can do every text encoding but it is fiddly to actually get it right.
Russian for "I Am A Cossack!", more or less I have a bad attitude and fear no one.
The mojibake would be trivial to fix but my position is that I should not have to, I leave it there in hopes someone at Apache or Mozilla will clue into it.
I've seen that very same character sequence whenever a Slashcode site is asked to handle Unicode out of the ASCII range. If I had to bet, I'd say that textwrangler fucked up your document somehow, whether through poor design or improper operation. It's strange that you'd think that someone from Apache or Mozilla would:
a) Be randomly reading your online resume.
b) Think that the mojibake contained within was the fault of some software that they maintained.
Additionally, I've had occasion to work with many, many people. I've found that -regardless of brilliance and competency- folks with bad attitudes almost always make bad coworkers. :)
Anyway. I hope you come to understand why your declarations about your side projects tend to mystify people, and why your strongly held stances tend to not gain traction with others. All the best, man.
People believe all sorts of things that are incorrect.
Carefully inspect -in a good text editor- the files that you have asked your web server to serve. I suspect that you will discover that what is being delivered to the browser is exactly what is in the files on disk. :)
Or, view source on your personal page. (Notice that the page is served up as UTF-8, so that what is served up will be what was on disk.) Here is a snippet of what's served up, verbatim:
<p>Español Mexicano, Español Castellano: "MEE-gə-LEET-oh".</p>
<p>Por Favor? Español Americano Centrale? Español Americano del Sur? "mə-LEET-oh".
"G" Silencio!</p>
<p>Я name is Міша. Я казах!</p>
When google has an opening they post it on their own site.
My index provides three items of value: a list of tech employer organized geographically, links to their jobs pages and link to their contact pages.
Its not always obvious what businesses are in one's city. Quite commonly jobs are posted on corporate sites but the jobs section of their own sites arent linked from anywhere. I often have to guess:
Thats because I dont regard craigslist job postings as useful to anyone.
I built my first index just for santa cruz county, on north monterey bay, to assist my coworkers in bailing from live picture because kate mitchell got the bright idea to shorten her commute by moving our office to san jose.
Just before our announced IPO, there was a reverse seven to one stock split, Kate resigned, live picture declared bankrupcy.
See, but your statement is super misleading. You introduced your jobs site with a direct comparison that all but directly claims that your site serves the same function as CL:
"Craigslist charges for job posts in competitive markets; the posts disappear after thirty days.
Instead, your site is effectively a rolodex of companies who hire tech workers.
While this isn't useless, it's completely unlike CL. Frankly, I can see almost noone[0] spending money on a Rolodex entry. I would regard anyone who did spend money on such a thing with a huge amount of suspicion.
And, uh, I've gotten a couple of really good, solid job offers through CL jobs postings, so they were quite useful to me.
[0] "Almost", because people spend money on unreasonable things all the time.
Yes I too have found good work through craigslist, but cl is far less effective than were the yellow pages in the phonebook.
Many companies never pay to advertise open positions, instead they post them on their own websites, then do not understand why no one applies.
I have received the greatest enthusiasm from residents of san luis obispo, california. there is lots of tech there but before my index no one knew how to find it.
If you can't follow my argument, then you need to ask questions of me so that I can better explain myself.
> ...cl is far less effective than were the yellow pages...
I find it hard to believe that -for all but the best of the best- cold-calling companies in the hopes that they have a job for you is more effective than contacting companies that have indicated their need to hire someone that possesses a skill set similar to yours.
Job hunting is hard enough. Cold calling every company that employs programmers seems like it is its own circle of Hell. Maybe you're just so good at what you do that people clear a space for you whenever you come knocking, regardless of their current staffing situation. For the rest of us, cold calling is a colossal waste of everyone's time.
> I have received the greatest enthusiasm from residents of san luis obispo, california. there is lots of tech there but before my index no one knew how to find it.
Uh. http://slo.craigslist.org/ ? Problem solved, since mid 2005 [0]. Job postings in that area are free of charge, too! :) ( To compare, your site has only been live since early 2014 [1].)
Not very familiar with him but stumbled upon this a while ago and it's some of the worst business and personal advice I've ever seen: http://craigconnects.org/2014/12/knowing-when-to-keep-your-m.... I wondered if he has some screws loose after reading it.
As someone deeply interested in UNIX history, it's also revisionist and dismissive of hard fought wins by people who don't cower like he did -- real UNIX is incredibly successful to this day.. SysV (Solaris/Illumos), BSD, etc.
Thanks for posting this. The TL;DR version of this article is:
> I was asked for my advice, there was a clear cut right answer, that would have been good for the company and good for the future of the internet, but would have upset my bosses.
> Instead, I kept my mouth shut to help my career. This makes me "grown as a person" and "politically and socially clueful".
No, it makes you a jerk.
> btw, running craigslist is a "public service" and makes me a "philanthropist".
So you've given away the $400 million you made off craigslist, then? No? Then you're a delusional jerk.
Tangential to the main issue but Tech Dirt needs lessons on basic journalism. CFAA should have been defined at first reference. It's just bad writing. I had no idea what CFAA meant until I Googled it. I wish these kinds of publications would learn what a style book is.
Craigslist had to sue or otherwise anyone could scrap data from Craigslist, whether they were making "Craigslist more valuable" or not. Simple as that.
1. Why is that? I think the most relevant field of US law with that sort of provision is trademark law, which shouldn't have been at issue here at all (3taps and Padmapper were not pretending to be Craigslist or authorized by Craigslist). Copyright law, which is sorta relevant, does not have such a provision. If your song gets pirated, you can sue only the pirates named Fred without losing your right to sue. The CFAA, which is definitely relevant here, does not have such a provision: JSTOR does not have their hands full with CFAA cases against everyone who ever sent an article to a colleague or friend.
2. What prevented Craigslist from giving a straightforward license and API to Padmapper? That would have sidestepped any such concern, without setting a precedent that everyone else must get API access. I think it's pretty clear from this case that they did not want to see Padmapper exist, and regardless of whether they were obligated to sue and try to shut them down, they clearly wanted to sue and shut this particular application down.
I don't know why this was downvoted, but this was obviously the case. CL had no interest in these 3d party sites. It's their data and their terms of use, so there really aren't any debatable points. Nobody has a right to CL data unless they explicitly provide that right. If I were CL, I wouldn't have give Padmapper an API either; their arrogance was astounding. They had no respect for the CL terms of use. They built an entire company around a knowing violation of their data source's terms of use. The Terms said, "you can do X," Padmapper did "X" then complained when CL said no. It doesn't matter what X was, Padmapper just knowingly did it anyway. They had the intent to violate the Terms of Use.
It's amazing how wholesale scraping of data was being celebrated, because the scrapers "were making the internet better".
This is the first topic I've seen were the HN's usual Libertarian bias disappears and no-one challenges the underlying notion that CL is seen as part of the Commons. Channelling Dagny Taggart, I'd say 3Tap & PadMapper were a bunch of moochers.
You're making it sound like the scraping was being done to replace the functionality of CL, which, yeah, would be pretty transparently shitty to do. But they weren't doing that, especially PadMapper: They were indexing the content to make it more accessible, an action that's been taken probably trillions of times and is pretty much the main reason most of the internet is even usuable today. It's like accusing Google of plagiarizing your website because they linked to it.
That's a good point - does craigslist have a robots.txt to prevent Google from crawling it? If not, isn't Google guilty of the very same thing, by aggregating the information via search results?
Craigslist doesn't prevent Google from crawling them. Not only that, Craigslist also sued at least one company for scraping Google results in order to index Craigslist postings.
Yes, I'd agree with that. That's why I think it's incorrect to claim that Craigslist had to sue, even if there was some legal obligation (which I don't think exists). They clearly wanted to shut Padmapper down. If they didn't want to shut Padmapper down, they would have given them a license and API, and then any legal obligation would have been irrelevant, because Padmapper would be an authorized user.
I don't think people are really arguing that part. Well Techdirt is a bit with the whole whining about "b-b-but 3taps was making CL more valuable" as if that mattered (apart from being open to interpretation).
The issue is that CL and this judge have set a precedent (I think?) that private companies can make up felonies by deciding what "authorized access" is.
You wanna sue a business for scraping your site? Hey, fine, that's a civil business issue. (And really, there shouldn't be legal limits on that kind of behaviour to a publicly accessible system anyways. Or at least there should be some level of malicious intent required.) Trying to use a law with criminal/felony penalties is grossly out-of-line.
> there shouldn't be legal limits on that kind of behaviour to a publicly accessible system anyways.
Are you sure about that? So you don't want any legal recourse for someone that overloads your API/web servers with thousands of requests per second? You might say a technical solution (like rate limiting) would do - but that will cost you time & money with no upside, and can be easily circumvented (use many clients in parallel with the same effect - hammering your server and increasing your costs)
I think a law for that is fine, as a last resort. It will keep people at their best behaviour.
We likely are in agreement - isn't a rule against hammering a server 'a legal limit on the kind of behaviour to a publicly accessible system'? I was disagreeing with parent who said:
> there shouldn't be legal limits on that kind of behaviour to a publicly accessible system anyways.
The difference between "limit on that kind of behavior" and "limit on the kind of behavior" is critical.
A limit on red cars is a limit on "the color" of cars, but it's not a limit on "that color" when someone is pointing at a blue car.
So when "that kind" refers to sanely-implemented scraping, then there should not be a limit. There is a limit on "the kind", to restrict it to activities like scraping that aren't inherently harmful.
Unfortunately english isn't very precise: it's difficult to parse what MichaelGG was referring to by 'that'. I assumed he was referring to scraping (his previous sentence), but it's also possible he was referring to the CL case.
For the record, I maintain my belief that there should be a law that applies to scraping of publicly available sites.
> You wanna sue a business for scraping your site? Hey, fine, that's a civil business issue. (And really, there shouldn't be legal limits on that kind of behaviour to a publicly accessible system anyways
> You wanna sue a business for scraping your site? Hey, fine, that's a civil business issue. ... Trying to use a law with criminal/felony penalties is grossly out-of-line.
And the CFAA also happens to be the law that makes it a civil business issue, thanks to the 1030(g) section that was amended in 1996. Wikipedia has elected to make no mention of this section, but other sources do.
What a shit show. From the surface it certainly feels like this is just money moving from one entity to another with Craig sort of taking credit for donating the money. There is clearly more to this story that the journalist either isn't disclosing or didn't find in their investigation. It's not to say Craig doesn't have $1M sitting around to give, and I'm sure he'd back the EFF, but this whole thing doesn't smell right.
Quite odd that he'd donate money from a settlement to an organization that filed a brief _against him_ in the case. I wonder why he did that. Ah wait, I see, that was part of the settlement, the other businesses said they'd pay the penalty if he donated it to EFF and he agreed. Ha, that's something.
Perhaps EFF should earmark his donation in a fund specifically dedicated to legal defense against any future CFAA suits filed by Craigslist?
This is really silly. Reading the article, it seems that:
1. Craigslist donated a bunch of money to somebody (which seems like a good thing)
2. Craigslist sued some company for scraping their data and using it in violation of their Terms of Service. And won a settlement. (which seems like a good thing)
So in short, good job Craigslist. Why is the author so angry?
Because you didn't really read the article (or possibly the article isn't well written but I understood it)
1. Your order of events is wrong. Craigslist sued. Won. Then used the proceeds from the lawsuit to donate. This isn't money earned normally.
2. The problem is the lawsuit itself. Craigslist is abusing a law that is meant to deal with hackers intruding on someone's private network. Visiting someone's publicly accessible website and access publicly accessible data is not intruding on someone else's network. Chances are other 3rd parties will now start to abuse this law. i.e. This somewhat reminds me of companies suing people for summarizing a news article into a blog post and linking back to the name original article. In this case, I believe the 3rd parties were creating embedded Google Maps based on the addresses found on Craigslist listing (because Craigslist pre-lawsuit refused to do so) and linking back to the original listing.
TDLR Craigslist successfully stopped a company from fair use of their public content by accusing them of hacking into their network. Craigslist then donates some of the proceeds of lawsuit's 'damages' to the EFF.
But wait, you haven't added any new information. You've just re-stated the information in the article.
So yes, they sued before donating. And yes, they used the proceeds. But that makes no difference to the fact that both were good things.
Even reading your comment (or its sibling that was posted during the same minute) gives enough information to come to that conclusion.
Craigslist said "don't scrape our data". This other company scraped their data and used it to build their business, so Craigslist sued. That's exactly the world I'd like to live in.
> But wait, you haven't added any new information. You've just re-stated the information in the article.
Yes because from the little that you wrote, it didn't sound like you understood any of it. Maybe you should elaborate more from the start?
> But that makes no difference to the fact that both were good things.
You're wrong.
1. Craigslist killed two companies in the process. The worse part is that if Craigslist only listened to its users and improved their website, none of this would have even happened.
2. Now any company can stop anyone from fair use of their content. All they have to do is accuse them of Craigslists' definition of hacking under the CFAA. e.g. news sites can now potentially threaten companies like Google with the CFAA for listing their articles on Google News
3. Maybe if the scraping led to an outright copy of Craigslist's listing content I can understand. However, they were transforming that content into something more (something that Craigslist refused to do on their own site for years), kind of what Google does with search results. Moreover they were linking back to the original post (attribution).
4. Just because there's a term or clause in a TOS, it doesn't mean that it's legal.
5. The main result from this is to help maintain the entrenchment of the status quo which hurts overall innovation.
Really we should start here: are you familiar with Fair Use?
I disagree. You're implying that Craigslist was passive, and they were just watching from the sidelines. They weren't. They sued.
> building their entire business off of data that the didn't have the right to use.
This is our main point of contention, and we'll probably just agree to disagree.
> It may be that your personal opinion is that once a piece of text is on the internet, it's fair game to use.
imo it depends on how you use it. Unless I misunderstood something, if we had it your way we wouldn't even be able to write wikipedia articles based on data gleaned from online content even with attribution unless the owner gives permission
This is a deeply inaccurate summary of what actually happened, but it's understandable why you'd write it, because it's the same manipulative framing Techdirt chose.
Also not relevant here because Padmapper was taking data from Craiglist (and presenting it in their own format), and copyright only covers creative works, not raw data. See Feist v. Rural and friends. This was a CFAA case.
If parent wanted to say that all publicly-accessible web pages should be considered legally scrapeable without triggering federal cybercrime legislation ... sure, I think that's a reasonable point of view, and roughly the EFF's point of view in this case. If, in the process of scraping and reusing data, you are infringing copyright (or breaking a civil contract, or whatever), that can be dealt with on its own, not by criminalizing the scraping.
That Feist v Rural case is interesting. The Feist Corp were not able to copyright telephone listings. I wonder if the telephone company itself could have laid claim to their copyright though.
Craigslist has always been a sleazy company and didn't get serious about the violence that resulted from the postings on their site until massive public pressure. Craig Newark seems like someone who will put profit ahead of everything.
Craigslist is a very cool company. They are basically a communistic approach to the internet. I say all the more power to them. If someone can do better than free, go right ahead.
Have you used Padmapper? It was better than free. It was much nicer than Craigslist for the same queries, but -- because of Craigslist's size and entrenched position -- the data largely remained on Craigslist, despite Padmapper also having a free posting interface. That might be fair business, but it certainly wasn't a fair comparison of their products or philosophies.
Your analogy of communism may be entirely too apt, given how communism played out in practice last century. Sure, you got the overthrow of the capitalist economy, but if you were a tiny country who didn't really want to agree with Moscow on everything, sucks to be you.
I used PadMapper. It was fantastic, the user interface was so much better than CL. ...but its scraping of CL postings was risky and morally wrong. The "fuck the rules" approach has worked for Uber and AirBNB, but it's not going to work everywhere.
Why is it morally wrong? It may be a risky technical decision, but how can it be morally wrong? Would it have been better if the data was processed via a client-side browser plugin?
And Uber's "fuck the rules" are on a totally different level than sending HTTP requests to a server to aggregate publicly accessible information.
1. The owner of copyright determines how the product is to be used
2. Provider of a service can set out terms of service
3. When using someone else resource, it is polite to do as asked ("Please don't finish all the staples" or "Please don't use my stapler John. Jane can still use it")
Increasing CLs server load & increasing their bandwidth costs was morally questionable, it became clearly immoral once they knew CL didn't like what they were doing.
> The owner of copyright determines how the product is to be used
I'm not entirely sure I agree with this part. If I say, "John's website is at this address," John doesn't get to say "HEY! I get to decide who links to me," because copyright isn't for facts, it's for content. That's why I disagree so vehemently with the PadMapper decision: All they were saying was "There's a CL post for this address." That's not content, and it's certainly not enough content to replace the functionality CL sued over.
> 1. The owner of copyright determines how the product is to be used
Padmapper mined data (apartment locations, prices, number of bedrooms, URL to Craigslist post), not creative presentation. Even if we believe that it's somehow morally reasonable for Craigslist to have copyright over the text of user postings, the data Padmapper required was clearly not copyrightable. This was a CFAA case, not a copyright case.
(Yes, in practice Padmapper also displayed descriptions and photos, but they could have avoided doing so, and the service would still have been useful.)
If we're talking about morals, then the fact that CL only got copyright via a click wrap assignment, and that most postings aren't copyrightable because there's no creativity, and that the actual creators (posters) benefited from this access, I don't think things held up.
I'd also be surprised to find a real increase in load, if the scraping service served end users. Otherwise those users would be requesting directly to CL. And, if the 3rd party service worked by riding on the back of users (say, via a browser add-on), would that suddenly change things?
Communist as in it's all mine? It seems like the means of production here is the data they 'publish' and they've been very clear who the owners of that are -- craigslist and no one else.
Their aggressively litigious nature makes it difficult for others to compete and provide better services. It's difficult to penetrate the network effect juggernaut if you can't scrape their existing ads.
For a while they even changed their terms of service to give themselves an exclusive license to anything you posted on the site in order to help them sue websites that let you search it without their permission - so in theory they could've sued their own users for posting the same ads on Craigslist and elsewhere. That's fucking insane.
"Aggressively litigious nature"? Really? Because this is the first lawsuit I'm aware of involving CL, and they were clearly in the right here. So ... what's your beef?
I don't know the specific facts of this case, but the communistic element has been at the center of some similar cases. Anyone is/was allowed to use Craigslist data to make apps as long as they aren't doing it for profit. And every year people agree to these terms, but later get their access yanked when they decide to start running display ads or whatever.
"All you posts are belong to CRAIGSLIST" is not very communistic. Craigslist is just a private business using government IP law to prevent potential competitors from eating their significant market share.
> "If someone can do better than free, go right ahead."
It's hard to compete with an entity that uses the coercive power of the state to squash you.
Just tried out nextdoor.com. It tried to load 6 different tracking and marketing scripts that my browser addons blocked. Craigslist has 0. I will not be using nextdoor.
It's nothing compared to what Google, Facebook, AirBnB, Uber and literally every other website you visit or email you open does.
Craigslist may not track your actions, but it doesn't ever update it's UI/UX or add features so there's no reason for Craigslist to collect user behavior data.
Now I completely respect your personal choice, but for others who are more willing to try out something that can provide them real value, I think Nextdoor will win over Craigslist every time.
What really amuses me is that in the Tweet highlighted, Craig is indeed implying that it is his money going to the EFF - who Opposed him in the legal case that lead to the money. So he should be called out for egregious bullying - using the threat of Criminal Sanction to get his way.
If an HN-friendly business (whatever that is) pulled the same move it would also be equally worthy of opprobrium.