Hacker News new | past | comments | ask | show | jobs | submit | mwmanning's comments login

Hey since this is blown up I just want to address it directly.

I take responsibility for what happened here. My RubyGems.org account was using an insecure, reused password that has leaked to the internet in other breaches.

I made that account probably over 10 years ago, so it predated my use of password managers and I haven't used it much lately, so I didn't catch it in a 1password audit or anything.

Sometimes we miss things despite our best efforts.

Rotate your passwords, kids.


Wow, that's a pretty well executed and possibly targeted attack then. It blows my mind how easy it can be to perform a high impact attack by abusing popular libraries. Hopefully this was caught before it got into production in high profile implementations.


Yeah I'm assuming the methodology is:

1) Find high-value target libraries

2) Grab the usernames of accounts with push access

3) Check those against password dumps

I feel really stupid about this, but like I said it was an oversight. I apologize and will try to do better.


Sounds like rubygems and other registries like npm should try to get ahold of those password dumps and check them against their own account databases somewhat frequently!


If you find a reused password, how do you let the user know though? If I got a "your account is vulnerable" message I'd ignore it as junk like all the other ones I get pretty much daily. You could force a change next time the user logs to your interactive interface, but many users won't do that for some time.

The best approach is probably to disable the account completely until an interactive login is made and a password reset can be forced but some would be up in arms about the inconvenience caused: you can't just allow a simple reset as the login could be coming from an attacker not the original user, an extra channel will need to be used to verify the identity. You might just have to leave the account locked forever and expect the user to create a new one - but now you have the old account and its content which may be used as a dependency of many projects which now break, unnecessarily if there hasn't been a login by a nefarious type.


You could send that notification, invalidate any client tokens, and also disable the compromised password forcing the user to re-authenticate through their email address, a-la password reset, and I guess also verify they aren't using the same password again.

You wouldn't lock the account forever, the point is to establish that the person whose password was compromised knows, that the password is not the only factor which is used to regain access to the account, and to ensure that your service (rubygems) and its downstream users are not compromised as well as a result of the breach.

Any groaning about the inconvenience caused by disabling account access until the password is changed, can be simply shrugged away in favor of security concerns, with a link to this story about rest-client.

By the time you have learned the user's plaintext password, their account may already have been compromised. There's a case to make that you disable all downloads of any gems that might be compromised from the account until you've verified they aren't. That might be over the top, especially for popular projects as now we are talking serious inconvenience affecting potentially thousands or more of downstreams.

It's a sticky situation, since you don't really know how long that password has been in the open for hackers to use and abuse once you've discovered it in a password dump.


Heroku did this about a year ago. They have a list of known pwned passwords (probably haveibeenpwned, but honestly I'm not sure), and disallow accounts to use those passwords. When that change was implemented, any account using a pwned password had that password expired.

https://status.heroku.com/incidents/1625

(source: I work for Heroku Support)


If a gem maintainer is re-using a known-compromised password they have absolutely zero right to be annoyed at the "inconvenience" of having to reset their password to something that isn't compromised.

RubyGems has a responsibility to its users and community here. It (like npm) needs to take this stuff seriously.


In other sites I know that actually implement this, they simply lock your account/force a reset so you can't login with the existing credentials.


Presumably you'd use whatever procedure you use for a lost password?

But simply forcing a password change at the next login after detecting an insecure password would not unduly burden anyone and would be better than doing nothing.


> but some would be up in arms about the inconvenience caused

Sometimes you have to have your priorities straight. If you found the password, someone else can find it.


Glassdoor emailed me this week with such an email. We found that your password was leaked, we have disabled your account and signed you out of all devices, you need to create a new password to login.


That's not very practical if salted hashes are being stored.


but the salt is in the database. You can hash all the known-compromised passwords with the salt and see if any match.


The salts should be different for each user, specifically to deter brute forcing of this nature.

The only time you would have access is when the user logs in, so for rarely logged in users you would have to proactively reset their password or cross your fingers.


Hopefully you don't transmit the password and are doing challenge/response so that you don't even have it when the user logs in.

But even with 12 round bcrypt hashing, you should be able to fairly cheaply attack a list of 2,000 bcrypted passwords with a million-entry database of leaked e-mail/password combos in a GPU-month.

Probably easier to force a password reset on everyone and then do the checking on password change, although you need to be careful there not to be sending the password.

EDIT: uhm, wait, so if you've got the e-mail address in the dump then there's only one user for that, so just grab their salt and hash the password and check it. So that million entry database should be checkable in a bit over half an hour...


> Hopefully you don't transmit the password and are doing challenge/response so that you don't even have it when the user logs in.

Wasn't challenge/response / SRP authentication debunked ?

https://www.nccgroup.trust/us/about-us/newsroom-and-events/b...

https://news.ycombinator.com/item?id=2859470


Make 2FA mandatory. Apple did just that recently with their app stores. Someone authoring libraries should be able to handle that.


It happens. You've taken reasonable precautions to safeguard your online identity, which is all one can really ask. Sometimes things slip through the cracks. The hacker is to blame, not you.

The larger question is about if gem/npm/cargo-style package managers are such a terrific idea in the long run. The security implications are pretty serious.


I doubt the initial attack was targetted. That would have been a brute force testing-known-passwords-against-similarly-named-accounts. Once a useful account was found it could well have been sold on the appropriate black market rather than the finder using it themselves.


It can happen to us all.


> I take responsibility for what happened here

that's.... rare. Well done.


Hi, Matt from Convox here. I'm happy to answer any questions you might have about this feature, Convox, or development workflows using Docker.


Thanks, Matt. This is timely but I don't have time to dive in right now.

Is convox-examples/node-workers up-to-date and pertinent to the new features?


That repo could use a refresh. convox-exmples/rails is most pertinent here.

We're working on revamped examples for many popular languages now and expect to see them rolling out soon.


The base cost for a rack is about $85/mo. Convox is extremely cost effective for small businesses and up, but would be expensive for running a single, low-traffic hobby app.


Is that $85 for small instances or micro? I noticed it defaulted to small, which is why I ask.


That's for the default of t2.small


That's exactly the etymology. We were thinking about datacenters and servers. The original name was convox/kernel, but we thought it wasn't very descriptive.

The naming collision with the Ruby interface is unfortunate, but we figured it was pretty easy to tell them apart from context.


Tbh I agree that as a descriptive name it works quite well.

However, it will muddy the waters for people trying to solve problems in future. For example, this tag on StackOverflow: http://stackoverflow.com/questions/tagged/rack


Hi, Matt from Convox here.

I'm happy to answer any questions you have about Rack or any of Convox's other offerings.

To join a discussion with our user community please join our Slack! https://invite.convox.com


What's Convox's business/funding model? Not skeptical, just genuinely curious and can't tell from browsing the site. Also, what are your other offerings? If everything between AWS and my site is free, what else is there to offer?


We were in the Summer 2015 YC batch and we've taken some VC funding.

Our other main offering is a web console for managing your team's access to rack(s) and integrations with 3rd party services. https://console.convox.com We charge a subscription fee for that and we also sell support and services packages.


Convox (YC S15) is hiring a Solutions Engineer.

Convox is an open-source, private PaaS that runs in your own AWS account. http://www.convox.com

You can read the full job description here: https://www.dropbox.com/s/g0o2dv9j7nsg8gi/ConvoxSolutionsEng...


Hey this is Matt from the Convox team. I'm happy to answer any questions about Grid (what it is, why we built it), Convox, or AWS/Docker in general.


What's your feelings about cedar stack being over 1.5GB in size after building a docker image?

I came across your guys' stuff about a month ago and was intrigued by the cedar stack of heroku being dockerized and started playing with it.

Immediately stopped when my simple hello world node app was 1.5GB in size.


The cedar image is admittedly pretty large. There's a lot of stuff in there.

Building a much more lightweight image for your node app shouldn't be too hard, though. Check out https://github.com/convox-examples/express for a simple example.


Yeah I was looking at kind of making a heroku setup on my servers sorta like Dokku, but I'm not a real big fan of Dokku. And I wanted to use CoreOS and Docker, but I realized that I can't just include all possible programming languages into a container to be built. So I've opted out of that and just build custom containers based on the app I am deploying.


Hey, Convox cofounder Matt here. Great questions!

1) You can update your Convox rack and cli using the cli. `convox update && convox system update`. http://docs.convox.com/docs/updating-convox/

2) We're working on an RDS provisioning service today and hope to release something this weekend. We're also in early talks with other addon (we call them "services") providers. For now if you want to use one of these services you can connect to them using environment variables. http://docs.convox.com/docs/environment-variables/

3) Our RDS service will eventually handle backups and recovery, but those features aren't implemented yet. We're also exploring partnering with other hosted database services that already have these features.

4) Convox is open source https://github.com/convox and totally free. You can use it however you want and you only have to pay for the AWS expenses. We're also experimenting with managed hosting, which we'll charge a small premium for, so please let me know if you're interested in that.


Thank you!


You could run Wordpress today, afaik. Anything that can run in a Docker container can run on on Convox. You'd just need to use some sort of hosted database service like Amazon RDS to persist your data.

edit: Just looked at Wordpress a little closer. I didn't realize it expects a persistent filesystem, which I guess is what you're asking about. Convox follows the 12factor.net philosophy of ephemeral filesystems, so you'd need to figure out a way to persist files to S3 or similar...


Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: