On: Introducing Twitter premium APIs

[andypiper]

That's categorically not the case - the index for the search API has always been around 7 days since Twitter acquired Summize (which is where the search API came from). There was never an artificial limitation such as the one you're describing. From the page you're linking to from the archive:

"The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets."

Scraping the Twitter website is against the terms of service and may result in permanent suspension of access.

[gboudrias → andypiper]

> Scraping the Twitter website is against the terms of service and may result in permanent suspension of access.

Scraping is always against the ToS, so no surprise there. I doubt it's really ever stopped anyone.

[oculusthrift → gboudrias]

i would imagine it stops researchers.. who usually try do things legit

[spaniard89277 → oculusthrift]

It does. We contacted them seeking academic discounts, and there wasn't. I was in a little university in Spain. No one is going to drop 20k like that.

[me_bx → andypiper]

My bad, my memory failed me, I thought remembering being able to perform search on the historical tweets via the search endpoint.

[keiyakins → andypiper]

So is using it to incite violence. The Twitter tos isn't worth the paper it's written on.

[jalfresi → andypiper]

Scraping is how the web works. After all, retweets are essentially manual scrapes

[cisanti → andypiper]

> Scraping the Twitter website is against the terms of service and may result in permanent suspension of access.

And what is Twitter gonna do about it? I never sign the TOS and scrape as much as I want.

[iamthirsty → cisanti]

> may result in permanent suspension of access

A.k.a.: they will try their best to IP ban you.

[b4lancesh33t → iamthirsty]

And then if you circumvent that, perhaps bring charges under CFAA.

[manquer → b4lancesh33t]

Would work only if you are based in US thou

[xgbi → manquer]

Yeah screw people accessing public information...

[iamthirsty → xgbi]

Well, that’s kinda the whole point. They don’t feel like it is truly public information. You gotta give something in return.

[williamscales → iamthirsty]

I wonder if this has changed in light of the recent LinkedIn decision?