Just a public service announcement-- for those of you looking to validate your own site for XSS exploits, you'll find that this list is particularly good at giving you "problem" strings to test against:
We honestly thought we secured XSS with common libraries on every field for one of our sites, but we paid a contractor to go through and use those strings in various unexpected places and they found a few special and unobvious spots that fell between the cracks.
Don't think you're immune just because you use all the right frameworks and input sanitation tricks. Even the best web developers are going to mishandle things from time to time.
If you think it's irresponsible for Twitter to miss some XSS vulnerabilities, you better be doing this testing on your sites, too, before someone else does and calls you out on it. If you're not testing at least the easy user input exploits, you don't really know how protected you are.
This is good advice, but OTOH, I do think it's going about things the wrong way to a certain extent.
Testing your site with certain strings isn't a complete way to determine if it's safe or not. You should "prove" to yourself it's safe against all attacks. Not just the ones you know about.
Definitely worth trying the "problem" strings as a simple sanity check though.
It's clear from the source that a) is vulnerable, whilst b) is not. I think programmers just need to be trained to have massive alarm bells go off in their head when they see some external string used in the output/markup in an unsafe way.
My point is exactly yours, but with added testing:
* Do the Right Engineering(TM) to guarantee no XSS
* Followed by having someone else confirm your certainty.
If you only do the first step, I'm nearly certain your site will eventually be vulnerable simply because we're all human and it's so easy to make mistakes in user input handling.
Otherwise you can wait until someone else proves that you missed something (using similar easy tests). Maybe they'll be nice people who will tell you about it. Maybe they'll be mean and will make you look like a fool publicly. Or maybe they'll be evil and exploit your users without you ever finding out.
I would assume Twitter is using conventional ERB templates since their front end is written using Rails.
The ERB templates used by rails don't escape HTML by default--you have to write <%= h user_provided_value %> instead of <%= user_provided_value %> to mitigate cross-site scripting attacks such as this. It's pretty easy to miss, even if you know you should always escape user-provided data.
Django's template engine, for instance, automatically escapes strings (though I think this can be disabled). To not escape HTML, you must either specify it or a method producing said string can mark it as "safe".
Though ERB is great in that it's incredibly versatile--we use it to generate several types of configuration files--I would very much prefer for it to escape HTML by default when asked to render a string in the context of an HTML page generated by a Rails app. Form helpers and methods such as "render" could be modified to mark Strings that contain HTML you wouldn't want escaped as "unescapable", much like Django does.
The way it works now invites problems like this, even from competent programmers.
Even after fixing their db/cache/queue infrastructure to not fall over in normal use, they still can't handle a DoS attack at all! They peer exclusively with the highest-latency and least-connected Tier1: NTT/Verio (incidentally, so do fellow idiots OpenDNS).
The business side is in fantasyland, as shown in the documents leaked to TechCrunch. Evan Williams has no idea what he's doing -- Blogger was pure dumb luck, it had already effectively died when it was bought and resurrected by Google.
I don't know how Alex Payne keeps himself sane in that kindergarten.
Yes, I agree. That's why we've tried to assist Rails Core in reviewing a more comprehensive string tainting model. I've said for some time now that security needs to be institutionalized in frameworks in order for developers to be unable to make stupid mistakes like this one.
Every non-trivial app has vulnerabilities. One indication of a vuln is not "blatant incompetence"
I've responsibly reported my fair share of Twitter vulns over the years. In general Twitter fixed them quickly. (Except some CSRF/XSRF issue which was non-trivial and was when they had fewer resources.)
Dave Naylor was just irresponsibly publicizing something, which makes him somewhat of a jerk. Being a jerk is not necessarily a Bad Thing and of course we/I don't know the entire story here, but that's what it sounds like.
For those interested in the details: the vulnerability was caused by inadequate invalidation of URLs submitted by users when registering OAuth applications. There was a validation routine on those fields, but the very end of the regex used in that validation allowed any number of arbitrary characters. The test coverage included a variety of invalid URLs, but none that simply put a space after the URL and started a <script> tag.
Just goes to show that you can validate, test, and still not be safe. It took another set of eyes to catch this one.
Unfortunately, the publisher of this blog post did not responsibly disclose the vulnerability to us in advance. We found out about it through other channels.
We would have had the fix live earlier, but as several people noticed in the comments, we had a brief outage.
It's odd that you count completely on validation to catch problems like this, rather than escaping the output as well. Writing regex validations to prevent malicious input seems like a seriously error prone thing to be doing.
Agree. Not trying to give you guys a hard time, but Twitter has had several vulnerabilities caused by unescaped user supplied data being output without sanitization.
Obviously I'm on the outside looking in, but IIRC rendering user pages is not particularly taxing on Twitter, to the point that they aren't cached. Would adding a simple sanitization routine have significant impact?
Yeah. "We let Twitter (via Kevin Rose) know about this before it went live but we think somebody saw the Test!" is a total numbnuts disclosure policy.
If one finds a vulnerability, follow the rfpolicy[1] or similar. Send an email to security@example.com, not Kevin Rose. (Seriously—Kevin Rose? What the hell does he have to do with Twitter's security response team?)
I agree that it was irresponsible for him to use his knowledge and disclose a vulnerability to the public before giving a heads up to the entity in which the vulnerability impacts. Infact, I am for non-disclosure of vulnerabilities due to the fact that this is how script-kiddies thrive.
I go a step further and assert that due to his marketing background, he used this stint for his own marketing purposes - I guess this can best be summed: to each his own.
http://ha.ckers.org/xss.html
We honestly thought we secured XSS with common libraries on every field for one of our sites, but we paid a contractor to go through and use those strings in various unexpected places and they found a few special and unobvious spots that fell between the cracks.
Don't think you're immune just because you use all the right frameworks and input sanitation tricks. Even the best web developers are going to mishandle things from time to time.
If you think it's irresponsible for Twitter to miss some XSS vulnerabilities, you better be doing this testing on your sites, too, before someone else does and calls you out on it. If you're not testing at least the easy user input exploits, you don't really know how protected you are.