"A cleaner API that's easier to use, with new developer features like the ability to specify which fields get returned, or retrieve more Tweets from a conversation within the same response
Some of the most requested features that were missing from the API, including conversation threading, poll results in Tweets, pinned Tweets on profiles, spam filtering, and a more powerful stream filtering and search query language "
I really doubt this is what developers have been yearning for in a revamped Twitter API. Previous APIs provided plenty of functionality, it's just that too many of us were burned by the ever-shifting policies around API use. I personally went from envisioning Twitter as the social data pipeline of the web to vowing never to touch it again within the span of one year. I can't believe this announcement wouldn't touch on policy and developer relations at all.
Actually, this was a major point of API failure when I last tried to use it. Did you know that, with the current API, there's no way to retrieve the replies to a tweet? This makes it impossible to build tools that, say, allow you to visualize a tweet thread, because you simply can't fetch the thread.
So I'll say that, as a casual, social, and informational user of Twitter (i.e. not business, not adtech, not academia), this announcement gives me a little hope that we might see a resurgence in 3rd party apps that provide more powerful ways of interfacing with Twitter. At least, that's my hope, maybe I'm naive that they'd really allow this.
> This makes it impossible to build tools that, say, allow you to visualize a tweet thread
it is possible -- it unfortunately happens to be part of my job description to do so
one of the sibling comments basically has it right. you can build it from the bottom up (where "bottom" is the deepest tweet in the thread you know about, one way or another).
let me tell you though, i am _very_ excited for this new API, because there is some absolutely horrible code i will be able to delete as a result of it
Conversation threading is huge. I've had to use a few tools that keep getting broke to try to manually capture replies and threads to tweets for academic use. (https://www.exquisitetweets.com/ and https://treeverse.app/ which seems broken now)If a tool comes out that can do this easily (I'm no programmer so command line stuff if a little beyond me) then I will be very happy.
I'm not sure which hack Tweetbot is using to do this (I think search as noted below), but you can tell it's a hack because it only works for recent-ish tweets. If you go to a tweet a couple weeks old and view details, you can see that it won't fill in the replies.
The way clients usually do this is by fetching all mentions to the user after a given timestamp (of the original tweet), and collecting only those that are replies to the tweet. Since there's a limit on how many mentions you can get, you might not get all the replies.
You have to use the search feature to look for tweets and then look at the parent tweet ID. This is what bots like threadreaderapp, etc do. You can construct it from the bottom up if you know the ID of the last tweet in the thread of responses too, but you wouldn't know that a priori unless the user gives it to your software somehow.
Speak for yourself. Their current API makes third party clients very difficult because they lack features and I hoped they'll announce something like this.
Don't get me wrong, I wouldn't mind them touching on the developers relations they want to build, but no company of their size will admit to wrongdoing in the past.
Also words aren't worth anything unless they come in a legally binding contract that can't change on a whim.
Given the examples of promoted API usage, it doesn't seem to have any backing away from their stance that they don't want third party clients. Add to that their API pricing was as I understand it not feasible for any new client app used by end users which gets any serious amount of usage and needs to move to paid tiers.
I wouldn't be surprised if you get an API that'd be better for third-party clients at the same time as all requests for API access for third-party clients are denied.
Most importantly none of these items, except maybe threads require a new API. Poll results would just be added to responses, pinned tweets would be a new flag to the search endpoint and a more powerful filtering and search language can just replace the search API. It seems weird that they are tearing it all down and forcing everyone to migrate.
Product support ticket systems use twitter and other social media sites to respond to customer feedback, as well as track and escalate issues. When you tweet complaints about your service at Comcast, there's probably a ticket created that someone from their support staff picks up. I think there's a perception that the quicker responses that these tools facilitate can help prevent negative feedback from going viral and doing broader harm to the company.
Right. And so, for those CRMs it is vital to not just catch the first tweet. Being able to catch a thread is important to catch the rant on the third tweet in a thread that explains the exact problem.
if they reintroduce user streams I'll be pleased, if they ease off the per-app user limit I'll be elated. it doesn't really mean anything if they promise such and such policies or initiatives otherwise we'd all be holding our breath for bluesky. what's important is what they actually end up doing, which I guess we'll find out next week
I was looking into doing something for my own use, and you can't even retrieve your own list of followers or blocked users in one go! You have to paginate it, each page is counted against the rate limit!
That's what killed off Falcon Pro. The author released a version of the APP with a secret way of putting in your own API keys and I used that for a bit. Then he released a new version, but couldn't just grant the old people access because it would quickly reach the rate limit.
It was about that time I just totally abandoned Twitter.
Use Mastodon/Pleroma/Misskey. Fuck lock-ins. ActivyPub forever!
I don't understand what is the issue if user can add their own keys - just release the app with default set of keys or even without and let the user set their own.
Other than that, indeed Mastodon and co. are great. :)
No joke. I maintain a bot that interfaces with the Twitter API to take in tweets and post tweets from specific users in chat channels.
I read this blog post to determine a) what I need to do to port to the new API and b) how much this privilege of using Twitter will cost me, and it is mum on both of those aspects.
An interesting side effect of making your product ad-driven is what it does to the incentive structure of treating your product as a platform.
Twitter is just one example of many of companies who have created an API, asked developers to build on it, and then later burned the whole thing down.
Fundamentally, if you let developers build products that access your data in ways that users want, the very first they're going to do is remove the fucking ads. And, obviously, if that's how your company makes money, you can't let that happen. Given the choice between killing your ads or killing your API, you're going to kill the API every time.
Imagine running a casino and offering a "slot machine API". If you let the developers discard all of the losing spins, you're gonna have a bad time. This is essentially what an ad-driven company does when they have an API that lets developers separate the data users want (actual data) from the data they don't (ads).
Unless this is the idea. Never have worked with it before but my understanding (confirmed by many of the comments here) is that the current API is pretty difficult to work with. Rate limit the new API and allow less restrictive access through paid tiers.
Suddenly Twitter is no longer reliant solely on ad revenue and all of the stigmas associated with it in the current changing environment.
With this, they are no longer an ad company but a platform ala Google Maps/Stripe. Plenty of companies would be more than willing to pay for unrestricted access to Twitter for data analysis.
Ad revenue may take a hit but won’t disappear overnight
I'd love access to the Twitter API for research purposes for my writing, but I'm priced out for that purpose.
I can't prove it, but it feels like they only want big businesses researching their markets, and not people like me asking questions about Twitter itself - even though letting people do that would result in negligible additional traffic.
This might be controversial and I know people will say that they're a private company and can do what they want, but Twitter has positioned itself as society's de-facto official communication channel and I think the data on it should be a matter of public record at this point.
edit: Does anyone have experience with scraping tools and Twitter? Presumably it's hard to do.
I've been collecting Twitter scrapers recently with the idea that I might eventually do something with them. This one [1] seems most promising but I haven't actually tried it.
This is just my own vague experience, but I don't think this specific to Twitter, nor does it really have to do with pricing people out.
I think that public APIs in general are a thing of the past, mostly because of bad actors, security problems and scale. Look at things like Clearview AI, just slurping up every photo they can get their hands on. All it takes is one or two abusers and then everyone has to be limited.
The simple instagram API that I used to just get my own most recent post 3-4 times per day, now requires me to submit an application, with multiple size app icons, justification for my use, privacy policy, ToS, etc. All to I can fetch my most recent post from my own profile.
It's just ridiculous how every large website is clamping down on their APIs, but with abuse left and right, they don't really have many options.
Have you tried contacting Twitter about this? They've given free access to researchers in the past. They'll likely ignore you or say no, and if they say yes it may be more restrictive than you like, but it's worth a shot.
I ran a Twitter app for 9 years that they crippled, crippled, then eventually killed with policies. It went from “@ev loves your app,” to “your app is against our terms of service” in that time. My Twitter days are done.
These folks from Twitter might suffer from a major disconnection from reality, they burned several bridges of trust over the years and now they just write a small disclaimer about them discovering the hack from this week and the new API apparently does not address issues people had with it in the past (that were used to choke 3rd-party apps). Kudos to the developers working on it directly though, I suppose it takes a lot of effort and the managers responsibilities are not yours to take home but they need to actually regain trust before anything else, no matter how fancy the API is. Some kind of "LTS" or long term contract for the API versions would be a start.
Exactly. What's their goal with the new API? Will they continue to be hostile to Twitter app developers?
Years ago, they made a distinct choice to become a business/celebrity platform, and actively moved away from the utility it could have been. That move cemented my view of Twitter as a dead-end for technology.
Twitter is not as profitable as they want to be, so the corporate wants user to do something monetizable, but users prefer what can’t generate revenue and don’t favor whatever that do, but Twitter is a private company so they have to do it eve if it would only result in its irrelevance, and that will lead to even less profit, then
This disclaimer is hilarious, it's like some terrible catastrophe happened far far away from them. But in fact it's been days of deafening silence out of Twitter, and they are fricking ground zero!
Like to think I'm quite positive when it comes to companies putting things out there, but still can't bring myself to trust Twitter's actions when it comes to their API.
I built and sold (albeit for a very tiny sum of money) one of those text-extender mini-blog tools back in the day off the back of the API. It felt easy, clean and part of the ecosystem.
Every action they took degraded that trust. I know they're a business and the data (and primarily their advertising serving and monitoring tech) is their lifeblood, but... fool me once...
One amazing use of the twitter API is to be able to mass delete your favs, rts, and tweets. It's the only thing I use it for as the API and platform as a whole is otherwise hostile.
Yes, the API can only get the most recent tweets but the script above can be used with the tweets.csv file provided within your downloadable Twitter Archive. You can delete tweets of any age via the API.
I had to use the archive initially to delete most of the old stuff but now I never have more than a few tweets to delete.
You're technically right, but the Twitter API won't return the next 3600, even after deleting the first ones. The response will be empty, even if you know for sure there's more.
No, my understanding is that Twitter has a "bucket" of the latest 3600 tweets you've posted / liked. This bucket is faster than their database to query, so once you empty the bucket, the API returns no result, and the bucket is never "refilled" from the database.
Those are not technical terms, sorry, but this is how I currently understand it so it might help you too?
Rather than “buckets”, I bet they just have a flag on each tweet in the database for whether it’s deleted. So if you delete the most recent 3600, next time they do a query to see recent tweets, it retrieves 3600, but they’re all deleted, so it filters them all out. Buckets are a little less likely implementation wise because you’d either sometimes see that the most recent bucket had very few tweets in it, or they’d have to be constantly shuffling tweets between buckets.
It is probably true nobody who currently codes against Twitter APIs wants new APIs. I've rewritten them before and won't be doing it again. Maybe someone new will come along to replace us. Twitter doesn't care unless it makes them more views.
Doesn't matter anyway because they already got rid of the useful APIs, which broke everything anyone I knew had written. Handwriting has been on the wall.
Sort of like boycotting Facebook, it's only takes the step of stopping to set you free.
As a user I would like for Twitterrific for example to support polls and pinned tweets. So at least one user of a third partly client wants this, I bet that there are more.
They are pushing stuff on the public API that has been available for years to “managed accounts” via Gnip.com.
This makes the paid APIs more widely available - it’s now possible to start using paid APIs without talking to a sales person, with lower budget (but not lower prices - some endpoints cost nearly $1 per call).
They also plan to remove free endpoints 6 months to 1 year after a paid equivalent is released.
I'd care more if I could actually appeal my dev account being perma banned for real reason other than I failed to clearly express what I wanted to do with a app
> You cannot create additional apps because your developer account was rejected or suspended.
I got banned cause they didn't agree with what I wanted to go and I couldn't email/message/discuss with anyone why my use was not approved and allowed. I couldn't resubmit with changes to the plan and I had a proper business plan and it all went in the bin as I couldn't get API access. Shouldn't of created it all on my personal account I guess but it was an important lesson on not relying on external services
Personally, I don't have an issue with bots on a platform, as long as they are there to be helpful. In the case of Twitter, I would restrict bots from being able to reply to tweets^, retweet, or like. Only output original things (like automated messaging) or restrict it to DMs. And possibly restrict the use of @s and #s in those tweets so they can't affect trending. That way you can seriously cut down the signal to noise ratio and also make it much easier to spot a bot account.
And please put a BOT label on every bot account and tweet they send. This should be a hard requirement.
^ However, this would interfere with bots like the threading or videothis types. So maybe not apply that restriction.
Nothing to see here. This isn't an announcement, it's a pre-announcement. Empty PR, "We actually love developers (despite our sordid history), yadda yadda". Almost all of the details are TBD.
When you login with the Developer account it gives you an option to upgrade and there you can see the prices. But I'll save you that: 2500 requests against their Full Archive costs 1899 USD. That's 75 cents per request.
That sounds absurd to me. 2500 requests is nothing and what if you're testing? The sandbox level is 50 requests for free.
Maybe I don't understand well the use cases but all the things that come to my mind for these APIs would required more than 2500 requests to do something interesting. They are pricing their data like if it was some sort of unique canonical truth vault when in reality 95% of it, it's just a bunch of shitposts.
Are you getting the prices from this [0] page? I think that's the pricing for the old API, because it still hast the standard/premium/enterprise split.
> In the past, the Twitter API was separated into three different platforms and experiences: standard (free), premium (self-serve paid), and enterprise (custom paid). As a developer's needs expanded, it required tedious migration to each API. In the future, all developers — from academic researchers to makers to businesses — will have options to get elevated access and grow on the same API.
> (https://blog.twitter.com/developer/en_us/topics/tools/2020/i...)
I hope they allow for blocking more than one user at a time at a minimum. Between the rate limit and only blocking one at a time, it takes forever to block a white supremacist and all their scumbag followers via the API. Much faster to just change your country to Germany.
For the free tier, the user endpoint users/lookup which now is just /users seems to maintain the same rate limit, and looking at the other endpoints I haven't seen something stand out as strongly reduced.
I notice that there doesn't seems to be a distinction in the new API between user context and app context as far as rate limiting is concerned, so maybe that would make a difference.
But a lot of endpoints are still missing so everything is still to see!
I always read comments about how Twitter is not profitable. Any ideas as to how much of their revenue comes from ads vs. businesses paying for API usage?
I get it, but anything with the phrase "new and improved" always strikes a nerve with me. In all honesty I hope other companies learn from this, and put in the effort to bolster their defenses-in-depth, especially at a CSE-level. I have witnessed many a times, how much power they are given, and how easily it could be exploited by just a bit of coercion.
Keeping my fingers crossed for removal of OAuth and other complications accessing the API. I hate using libraries for this, all I want is a simple token-based REST API, something like Facebook and many others have had for years.
Lots of (seemingly justified) complaints here. Is anybody on Mastadon? I have joined, but I have no followers and don't know who to follow. I'd love some connections over there.
"A cleaner API that's easier to use, with new developer features like the ability to specify which fields get returned, or retrieve more Tweets from a conversation within the same response
Some of the most requested features that were missing from the API, including conversation threading, poll results in Tweets, pinned Tweets on profiles, spam filtering, and a more powerful stream filtering and search query language "
I really doubt this is what developers have been yearning for in a revamped Twitter API. Previous APIs provided plenty of functionality, it's just that too many of us were burned by the ever-shifting policies around API use. I personally went from envisioning Twitter as the social data pipeline of the web to vowing never to touch it again within the span of one year. I can't believe this announcement wouldn't touch on policy and developer relations at all.