Hacker News new | past | comments | ask | show | jobs | submit login
BrowserStack was hacked
262 points by spiralganglion on Nov 10, 2014 | hide | past | favorite | 108 comments
I just received a very strange email from the support account at BrowserStack. I cannot, however, verify the information.

The contents of the email: http://pastebin.com/RQXd2Au3

Can anyone else verify if they've received the email, or what the official word is?

If there's any information from the email itself that I can provide to verify the authenticity of the message, please let me know. I do have a BrowserStack account that I use regularly, and as such find this email to be quite worrisome.




They just tweeted this:

>We did get hacked. Currently sanitising entire BrowserStack, so service will be down for a while. We're on top of it & will keep you posted

https://twitter.com/browserstack/status/531631012493524992


If they don't deliver some kind of post-mortem for this, the problem isn't likely to go away.


They're a little less forthcoming on their website:

"We’ll be back soon!

Sorry for the inconvenience but we’re performing some maintenance at the moment. If you need to you can always contact us, otherwise we’ll be back online shortly!

— The Team"


Rather than intentional deceit, this is likely their standard down-for-maintenance page that just hasn't been updated to reflect the current situation.


I'm not ruling out the possibility that their twitter was compromised, but this is looking bad quickly.


Standard post potential security incident rules apply folks.

Don't engage with suspicious emails. Do not attempt to access your account for now. If you have used your Browserstack password elsewhere, go and change it about the place. Watch out for any links from untrusted sources on this subject as they may be malicious.

Hopefully there will be more solid updates soon.


> Don't engage with suspicious emails.

You shouldn't do that anyway, regardless of whether there is an incident afoot or not.

> Do not attempt to access your account for now.

Why not? The horse has bolted. Just assume that it is compromised and that anybody can read over your shoulder. That's probably safest with a service like this anyway.

> If you have used your Browserstack password elsewhere, go and change it about the place.

No, if you've used your Browserstack password elsewhere then you should immediately stop that practice and use different passwords for all the services you use, not just change it everywhere else.

> Watch out for any links from untrusted sources on this subject as they may be malicious.

Links from untrusted sources can always be malicious, and should be treated as such. In general, if you didn't initiate the conversation you have to be wary and you should always verify the source of links in email or other communications (and preferably websites) before you start running their code (aka: clicking on their links).

If you are using Browserstack and if you plan on continuing to use them and/or their competitors make sure you interface with test systems only and make sure those systems do not contain any real (privacy sensitive) user data or other data that might lead to your service being compromised in turn, assume someone is reading over your shoulder at all times. This includes test scripts, especially scripts containing credentials, those should really only live as long as the test session.


Regarding some of the actual claims, I had an issue a year back when I logged into a session, and could perfectly see another user's session in progress, internal url in the browser, mouse moving around.

I freaked out, watched for 3-4 seconds, and then got kicked out of the session.

I opened a ticket with support, and they got back to me saying they had "fixed the root cause".

I still use browserstack, but I'm really careful with passing along private credentials.


I've encountered similar oddities, seeing the remnants of other users' sessions, but no mouse movements so I guess they'd been recently terminated.

Also the VMs aren't locked down as tightly as they ought to be, last time I poked around in the Windows ones there were a few folders left writeable that shouldn't have been, ones with executables and scripts in used to control it.

That said, their service is so very useful that I continued using them anyway. My use case is just to occasionally check some public-facing websites are rendering properly on various browsers, so no big deal if someone snoops on that.


I really saw mouse movement and url typing, even though it was only a few seconds.

I raised the issue with them back in July 2013, but was initially brushed off. A couple more aggressive emails and they finally responded after 2 weeks saying the issue had been solved.

Haven't had problems since, but still wary..


Sorry I didn't mean to sound like I was doubting your account, was intended as a confirmation of your sightings!

It's just as probable that I connected in while the users were idling.


No worries! Just pointing out the fact that they didn't seem too worried about the issue when first notified, and I really had to press to get some attention.

Pity I wasn't fast on a screen capture!


Same here, in Nov 2011. I contacted support, told them about it, and cancelled my account/sub right away back then -- I was using Browserstack for developing client sites and applications, and didn't feel like getting sued over a service I am using potentially leaking sensitive information.


that happened to customers at a company I worked for. Turned out to be awful handling of threading in a rails app caused sessions to get mixed up.


Is there a good browserstack alternative?


The only other 2 I could think of is SauceLabs and Rainforest QA.


Good. Probably no. But there is Saucelabs.


There is, actually; Crossbrowsertesting.com .. it's good.


I think browserling is a similar service.

browserling.com


This is why we at http://testingbot.com provide a pristine virtual machine every single time when offering browser testing.

After the test is done, libvirt destroys the machine to make sure nobody else can see what you did with the virtual machine during testing.


There presumably are independent process auditors that would come in and validate your claims about your systems. AFAICT Browserstack were claiming to be entirely secure too.


Hey! A founders gotta do what a founders gotta do! Nice timing.


Wow, that's just wrong... really?


How is it wrong? If someone was promoting their product in conjunction to some loss of life, or tragedy, that would be wrong.

A service is proven unsecure, so a founder provides his companies alternative product.


@defied is basing his promotion of his/her product on an email from a hacker. BrowserStack hasn't yet explained what happened exactly. That means that he/she believes the email from the attacker? This is just wrong. He/she is trying to use an event he/she might know nothing about.

Even huge companies like Google get hacked. So saying that you can do better just because someone got hacked is just ridiculous. It's about how the hack was handled and what information was disclosed that matters IMO. From what BrowserStack says, it seems like only a few emails addresses were disclosed. I'm not saying that this is bad. But this could have happened to any other company. Everything in the fake email is probably bull---- (has anyone tried to find out if anything in it was true?).


They've confirmed via Twitter this happened. If a company has a serious security breach requiring them to take their entire fleet of servers offline, I'd say it's prime time for advertising a competitor.


Yes, they did. But as I said, they didn't given much detail. But they did say that only a few emails were disclosed. If you call that "serious", then OK, but it could be much worse. The competitor is just profiting from something no one has any real information about and is basing himself on an email sent by a hacker: great!


Seriously!!! Marketing your product here.


If this isn't the best timing then I don't know what is.


You just lost a customer.

I refuse to use your service any longer after this shameful and down right unprofessional self promotion.

Testingbot is FAR from anything close to a perfect product and sometimes border line even a good product at all.


This is HK, self promotion is a core part of the ethos. The whole place is very natural selection-ee, survival of the fittest, and so on... It isn't like Reddit where self-promotion is seen negatively.

I mean you'll often have two or three startups posting around here all working in the same thing. So there is going to be self promotion.

In my view the above post isn't unprofessional. Why? Because they didn't criticise or pile-on BrowserStack, they simply said: "Want an alternative? We have an alternative! <link>."

Is it self-promotion? Yes. Is that wrong? No. It is unprofessional? Also no. Could they have been unprofessional? Yes: if they had criticised BrowserStack DIRECTLY in public.

Also you're a first-post throw away. Do you work for BrowserStack by any chance?


The port the pastebin claims is VNC. BrowserStack has a VNC repo and one of the two contributors to it https://github.com/browserstack/OSXVNC/graphs/contributors has the same github handle as the alleged VNC password.

Putting said nick and VNC into Google also finds emails from quite probably the same person to some VNC email lists.

I am not saying the pastebin is right but this makes one wonder.

Also, the VNC password -- at least by default tools -- indeed is stored in plaintext (see http://linux.die.net/man/1/vncpasswd "Note that the stored password is not encrypted securely"). It should be readable by the owner only, however.


We deeply apologise for the concerns that our users have been experiencing due to the attack on BrowserStack. We have determined that the hacker's access has been restricted solely to a list of email addresses. As a precaution, we recommend changing your BrowserStack password.

We are still in the process of sanitisation, and making doubly sure this situation never reoccurs. We are on top of it, and will post updates as they happen. Thank you for your patience. BrowserStack will be back up in a few hours.

-Snehal@BrowserStack


The mail was sent from Amazon SES. To be able to send email from your domain they had to verify it. TXT _amazonses.browserstack.com doesn't show any record for verification. How could the hacker that had solely access to a list of email addresses verify your domain?


Does anybody have the e-mail that was sent raw headers? Perhaps they sent the e-mails from code that calls out to Amazon SES?


We will be sharing an entire post-mortem in the next few days. Currently, all our efforts are focused on getting the service up and running and to ensure our users’ interests are taken care of.


> and to ensure our users’ interests are taken care of.

Every time I see that line I know that 'users interests are not currently taken care of'. If it were you'd be taking the GP a bit more serious, he's supposedly one of those users you're trying to take care of. If all you're going to do here is to say 'nothing to see here folks and we're on top of it' then you might as well say nothing.


That you're not saying "the passwords and other details mentioned in the email are bullshit" is concerning.


The tone of the email is very definitely meant to get the end user angry, this is not a true shutdown email or public service announcement of any company that expected to continue to exist / avoid lawsuits.

... Whether the company will continue to exist after this email is another matter.


Yea, I wonder whether there are companies that are really that honest, without the root pw and port numbers of course.


This email spells 'bad leaver' all over. Besides that, even if it is a bad leaver you'd hope that what's in that email isn't the truth but enough users of browserstack have at least partially verified the truth of some of the claims.

Browserstack is a very useful service, and it would be a pity to see them go. That said, if the claims in the email are true then they deserve to be replaced. Note how the email strikes right at the heart of the trust relationship between browserstack and their customers, that's a very sensitive spot for a company like this and it will take some iron clad and independently verified claims to restore that confidence.

In the end the email may turn out to be prophetic in that it will in fact cause browserstack to shut down.

The handwavy 'we're on top of it and we'll keep you posted' doesn't do much to reassure, they're clearly not on top of it (if they were this would have never happened).


Sounds like someone hacked their servers and this is how their version of disclosure.


Sounds like someone didn't like the terms of their severance and decided to hit them hard by exposing some very dirty laundry.


Sounds like someone is likely to never be employed by a software company ever again.


I got it as well, came via an aws account.

I filled in a support request with browser stack.

Seems very odd, angry ex member of staff maybe??


That was my immediate thought. This really does have the tone of a disgruntled (ex-)employee.

Edit: I just double-checked. Previous support@ emails were not sent via AmazonSES. This one was.


Their previous marketing mails were send using Amazon SES. Just like this one successfully signed with DKIM. Looks like the hacker had access to their Amazon SES credentials.


Yeah, looks like angry ex-employee(s)...


Seriously, how could ex-employees be so ignorant though? They are going to get taken to court, and rightfully so.


IANAL, but I'm at least curious whether they'd be willing to take someone to court when the discovery process would likely involve documenting claims of false advertising. Whistleblower laws might also apply. Will be fascinating to watch.


I don't think in India, where Browserstack is based, Whistleblower laws are that strong, the false claims could often be masked as a "flaw" and a bug rather than by design, and hence not sure how it plays out, but I think from the looks of it, if it is an ex-employee, that person is at a riskier position.


Curious as to how you're determining that...?


All BrowserStack services are now up and running. We are keeping a strong check on the system and will email all users the entire analysis.

Sagar @BrowserStack


I would probably still use them even if this information is true because we never had any important data go through their service (just testing accounts) and because I am not aware of any good alternatives.

... would look for an alternative first! But for now assuming that this is not real, anyone checked if it is real?


SauceLabs.com is an excellent alternative.

Every session gets a brand new VM and they have some great automation and manual features.


CrossBrowserTesting.com provides both live testing (over 750+ browsers), automated screenshots, and automated selenium / junit testing.

All VMs are destroyed after each use. It is more expensive to have to restart each time, but we feel it is the right way to do it, and it ensures a clean uncompromised configuration (disclaimer - I am one of the cofounders, so extremely biased :) )


I thought VM snapshotting facilities/copy-on-write made it almost trivial to start with a fresh virgin one every time. With enough RAM on the VM host the changes from one session wouldn't even need to be written to a physical disk.


We never write changes to disk, the code is trivial really. And it always launches from a locked snapshot.

What we are always trying to optimize is the time to the user screen. That is really the expensive part, once the disk reads happen we want a working session on the users screen as quick as we can get it. Or that is the plan anyway. :)


Currently attempting to convince our team to switch to you guys (not just because of this event) your service is sooo much faster!


Cool, thaaaaanks!


I've been using saucelabs for over a year. It's very comparable


Worth mentioning that Saucelabs takes security very seriously. If anything in this email is true (I can't speak to it at all), then I wouldn't use BrowserStack for testing anything connected to my systems. Sauce, on the otherhand, destroys every VM after each use, guaranteeing that you get a known good (and uncompromised) state for each usage. And unlimited manual testing is $10 a month, which is a great price.

(Disclaimer: Former Saucelabs employee. Currently doing other things unrelated and don't keep in touch, but I definitely know the Sauce claims in http://sauceio.com/index.php/2011/09/security-through-purity... are true.)


I've been looking into them for automated testing of our frontend, and their automated testing hours per day looks like the best bang for the buck for our price range.


Browserling.com is an alternative.

(Disclosure: I'm the co founder.)


Here come the articles:

http://www.neowin.net/news/browserstack-a-browser-testing-se...


Oddly enough they are currently down for maintenance...

We’ll be back soon!

Sorry for the inconvenience but we’re performing some maintenance at the moment. If you need to you can always contact us, otherwise we’ll be back online shortly!

— The Team


It was online around the time of OP's post. Looks like they went offline in response to this event.


Is the security claims in the email real? It should be possible to verify relatively easily -- and it should be done soon before they patch the security issues.


And hopefully the patch is more than just "looks like it's time to change the root password"; if the claims in the paste are even remotely true, they have deeper OpSec problems which will take longer than a brief maintenance window to fix...


I don't think there's anyway this is a real email. You'd think they have a little more dignity than this. Revealing ports and passwords is completely insane.


It's definitely not an official email, but that doesn't mean the content is not real.


Very true.


Do the passwords provided in the mail actually work?


You'd be stupid to try and find out.


If you (with some care) tried against your own instance, I don't think it would be too stupid. At the very least see if there is something listening on port 5901.


I don't think that is very fair. You would be able to test it out on your own VM, for instance.


On your own VM?


This sounds like someone got hacked.


I don't know if they're shutting down, they have to feel the heat somewhat from Microsoft's free IE testing service[1] though. modern.ie still has references to browserstack, but I wonder why.

[1] https://remote.modern.ie/


Too bad that they've partnered with browserstack!

>We've partnered with BrowserStack to bring you interactive browser testing on the cloud.


"you’ll need to download the RemoteApp client"

Microsoft still stuck in the 90s.


curious what would be a 2010s solution, I believe these apps run as if they are on local machine.


How bizarre, that seems like a disgruntled user; even if they were shutting down, they wouldn't word the email that way.


Prevention is better than Cure, So better to find alternative solution. There is a secure storage software called basefolder (http://www.basefolder.com)


Got that same email. Went to browserstack.com and found a maintenance sign.

What's going on?


Automate and Screenshot services are up and running. Live will shortly be up as well. We will be emailing all our users with the entire analysis of the attack soon. Thank you for your patience.

-Snehal @ BrowserStack


Don't want to overstate the obvious but this seems personally motivated. The whole password policy thing sounds reasonable, but don't most admins have access to pretty much everything?


There's no reason for admins to have access to temporary VM's that are created for the sole reason of running browser tests.

It also makes incidents like this much harder to happen in the first place.


doesn't surprise me. easy to open a terminal in their OSX vms and poke around. guessing someone a lot more knowledgable than me could wreak some havoc.


It should be secure to the point they assume all VMs are always compromised, especially given the risky climate online.


I guess that's the death knell for browserstack :-(? I am always worried that something like this happens to all those cloud providers out there...


Could you post the entire raw message headers? Interested to see if it was sent from Amazon simple email service.


Considering that they are actively hiring in their Mumbai office, I really doubt that they are shutting down.


It's hard to belive it's a legit email, more likely their system has been compromised.


Happened for us as well, definitely got hacked by some joe schmoe, or a disgruntled employee.


Back online now, hopefully some information will be forthcoming.


The browserstack website is currently down for me. Anyone else?


> BrowserStack is Shutting Down

While the email is certainly not legitimate, the subject may very well turn out to be true. Should a company which is indeed so negligent continue to be in business? I guess we will find out.


They probably got hacked. This happens to the best companies and nothing much can be done about it. I don't think it's fair to call them "so negligent" yet.


> This happens to the best companies and nothing much can be done about it.

I agree with the first half of your sentence, but not the rest. A lot can be done and many companies do.


That email looks like a very targeted email.


Man...this looks like insider attack. This much targeted attack is very rare to a company like this !. Hope they will resolve it as early as possible.


Well, if what the perpetrator says in the email is actually true, it looks like it would've been very easy for any customer to gain the same level of unauthorized access.


If this is a disgruntled employee, this has to be the stupidest move ever on his/her part. He will definitely be jailed for this


i just got this also. Hopefully this isn't true.


I just got an email that my account was automatically renewed, so I hope not!


I just got this too.


I have received the same email notification, and submitted a story to Slashdot:

http://slashdot.org/submission/3969603/browserstack-compromi...

Up-vote it (+) if you have received the email and you ARE a BrowserStack customer.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: