The real lesson here is that when you fix your mistakes, go back and fix your mistakes retroactively!
AM used an insecure login token at one point, and 3 years ago they fixed it. They switched from an MD5 of lower(pass)+username to an MD5 of the bcrypted pass+username, which is no longer reversible.
Apparently they never updated all of the previous login tokens though, so anyone who had created an account before the new secure system was put in place still had a vulnerable token stored.
When it comes to security, when you fix something - fix it for everyone people! Even if it's hard.
The good news for these folks is that the passwords revealed appear to be over 3 years old, and we all chang our passwords more often than that, right????
I thought the standard way of migrating your PW hashing function was that you could only do it during a login, because that's the only time you have the PW in plain text. No?
> I thought the standard way of migrating your PW hashing function was that you could only do it during a login, because that's the only time you have the PW in plain text. No?
No. Well, I mean, it's common to do that, but it's a bad idea and it's not necessary to keep all of those vulnerable hashes around.
Say you have a bunch of MD5 password hashes stored, and you want to upgrade your password hashing to BCrypt. Don't wait for the user to login and rehash their plain text password.
Instead, Bcrypt all the MD5s. The authentication mechanism is now BCrypt(MD5(plaintext)) == stored_bcrypt. Upgrade them to straight bcrypt on login.
> That doesn't really have the same security properties as the original login information.
Well, obviously. For instance, it lacks the property of being trivially vulnerable to a GPU hashing the entire dictionary with a given salt in 30 seconds flat.
No, you just take the original MD5 of lower(pass)+username and run that thru your new bcrypt. You will have a more complicated authentication process but it's worth it.
Yes, in addition to migrating to pure bcrypt/scrypt at next login. I've implemented this migration strategy for two different web sites (not listed in my HN profile) with success.
Not so complicated if you plan for this sort of stuff ahead of time, or even at migration time. You can prepend your hashed fields with the algorithm used, and have a library that automatically handles them.
This also makes generating test data easier, as our library has a "plaintext" hash type, so we can just insert "$plaintext$password" rather than having to run a hash on the password.
The entropy in the resulting hash is almost the same as a salted MD5 though, is it not? You have more bits, but not significantly more possible hashes.
edit: I guess most people only start out with short passwords, thus limiting the possible entropy anyway...
> The entropy in the resulting hash is almost the same as a salted MD5 though, is it not? You have more bits, but not significantly more possible hashes.
The entropy isn't the significant problem with MD5. The problem is how damn fast you can perform 2 billion MD5 hashes on modern machines.
1: Vulnerability to a pre-computed table. This hits hard if you are using plain-old MD5 without a salt.
2: Vulnerability to a newly computed table. This will hit you if you use a global salt and a fast hashing/encryption algorithm.
3: Vulnerability to high throughput custom hardware. This will hit you if you use a fast hashing algorithm like MD5. Modern GPU-based algorithms can churn through hundreds of billions of hashes per second on fairly inexpensive hardware. Which means that attacking, say, 20 million accounts it's possible to run through billions of possible passwords (which means a full dictionary attack, the top million most common passwords, plus common ways of combining words to make passwords, plus every random alphanum string up to 9-10 digits) for each and every account in only a month of work.
4: Vulnerability to very weak passwords. This will hit you if you use a slow hashing algorithm and there's no global vulnerability because you use per-record salts (or bcrypt/scrypt).
4 is where you want to be, always. You can't protect against weak passwords, but you can make it expensive for weak passwords to be revealed, and prohibitively expensive for stronger passwords to be systematically cracked.
The entropy of the hashes isn't really relevant since it's the repeatability of the hashing that's at issue. For example, the hash 7c6a180b36896a0a8c02787eeafb0e4c seems to have plenty of entropy, but since it's the MD5 of the string "password1" and you can just google that hash to find out that information, the theoretical entropy is moot.
Entropy is not really the issue here. For a dictionary attack with a non-reversible hash, what matters is how computationally intensive it is to generate the hash. MD5 hashes can be generated very quickly, while bcrypt hashes take much, much longer to generate.
You could delete the old hash and require users to generate a new one through the password reset mechanism. Generally you would want to combine both approaches, so that only users who do not log in during the transfer window have to deal with setting a new password.
Bingo. All they had to do is delete all the login tokens. Users with what would have otherwise been a valid cookie would have had to re-log in, but that's a minor inconvenience that's expected to happen from time to time.
Seems like a 10 minute fix at most to me to rehash the old password once the user was authenticated then put a flag whether they've been 'fixed' or not. After 3 months deactivate the old passwords and sent them an email reset.
Although AM was in the business of essentially blackmailing their customers. So ease of getting back in was of higher priority apparently.
Maybe another lesson is that security by obscurity is never going to work in this age. If someone steals your code, which is incredibly likely with the amount of digital infiltration happening constantly, your security mechanism still better be safe.
How do you verify that? Since bcrypt generates a random salt when you hash a value, if you MD5 the resulting hash, you lose the salt. So, are they storing the bcrypt work factor and salt but doing an MD5 only on the hash portion? Seems like a needless extra step. Storing the bcrypt work factor, salt, and hash should be sufficiently secure.
Updating them retroactively requires those users to log in again, doesn't it? I have had to do a similar update and you can't just update everyone's hashes retroactively. If you've properly hashed it, you need them to actually input their password again.
It's possible to update everyone's password without them logging in. If the old storage was just MD5(password) and you want to update it to MD5(bcrypt(password)), you're out of luck. You need the plaintext password to make that change. If instead you change your new storage to MD5(bcrypt(MD5(password))), you can take all of the hashes currently in your database and update them at one time with MD5(bcrypt(old hash value)).
The proper solution would have been to flag the old passwords and require everyone to change them on the next login. After a couple of months you null out the remaining passwords and require anybody coming back to go through the password reset system.
As far as I know, that is correct.[1]
Any change, whether for work factor or algorithm in code still waits for the user to come back and attempt a login, at which point you update the database stored attributes and the hash.
I think that if you can update a user's password without them inputting, then you have two problems.
[1]not authoritative advice.
edit: seems that comments suggest doing the md5(bcrypt(hash)) can be used to upgrade across the board
I thought that a while ago but then someone just pointed out that you bcrypt hash the MD5 hash and support two step (MD5 then bcrypt) until they login at which point you can rehash using only bcrypt.
Hindsight is 20/20. Working on legacy software devs sometimes want to implement the proper solution on paper rather than the best one. Perhaps it was a task that the dev didn't get enough time to write something to update the passwords of a few million users. Perhaps they thought they'd come back to it in a few months and force a reset on users that hadn't switched their password yet. Who knows.
For a non-native speaker, could you please confirm or invalidate my understanding of this interesting text:
1. They attacked some login/api-token unrelated to bcrypt.
2. If I use bcrypt-validate for logins and only temporarily associate rotating, random login/api-tokens with an account, I should not be prone to such attacks.
AM took the unencrypted password, lowercased it, and hashed it into an MD5 token that they then stored - conjecture is that it was used as a login token. That is what the article indicates was cracked, since MD5 is very weak, to get a lowercased password, then tried every permutation of capital letters on the bcrypted passwords, to get the actual passwords out.
To avoid similar issues, if you generate a token, don't use the unencrypted password as part of it. Random tokens are fine.
The title of the article should really be changed to
"How we cracked millions of Ashley Madison passwords by bypassing their strong bcrypt hashes because they thought they were clever" but that's less clickbaity
Also, never ever roll your own encryption - it will be flawed (unless you employ at least 3 crypto experts and get it peer reviewed - and even then it's probably still flawed).
They didn't really roll their own encryption right?
They rolled their own session login stuff, used it for some sort of login-key (what I assume they passed with server sessions). Why they would do that instead of a simple session ID I will probably never know. Maybe they did it so they could have independent backends (instead of having to share session keys across load balanced api servers).
> Maybe they did it so they could have independent backends (instead of having to share session keys across load balanced api servers).
Since at some point the loginkey was based on the unencrypted password that means that they either had to store the PW in clear (and fetch them from the DB) or generate the key at account creation and fetch it from the DB anyway.
A good strategy to check the token validity without reaching the DB is to sign it cryptographically. The loginKey can be $randomString+sign($randomString). The endpoint can check the signature before doing more costly things like a network request.
Yeah, I'm familiar, which is why I was surprised they had to deal with this at all -- most frameworks handle sessions like this for you automatically...
If you're going to signing, you may as well go all the way and just use an encryption + signing mechanism. Then you can store actual data in the token -- within reasonable transmission limits, of course.
That is a login routine/authentication. If they had replaced MD5/bcrypt with their own in-house alternative, that would be rolling their own "encryption" (hash function). But as is, they designed a login/authentication scheme that was highly flawed, but used off-the-shelf hash functions to do so (even if MD5 is deprecated at this point for anything security related).
To phase this simply: "If they didn't do maths, they didn't design their own encryption scheme." They did not, so therefore they did not.
Now you can argue the merits of using an off-the-shelf authentication/login scheme (e.g. Kerberos, OpenID, ASP.net's authentication provider, etc) and I would agree. But that isn't what you said, you specifically said they rolled their own "encryption" which they did not.
Well, lets be fair here. Saying "Don't roll your own encryption" doesn't specify "maths" or "applied crypto" (which incidentally covers security/auth mechanisms).
It's a bad idea to roll your own versions of either the maths part or the applied part, so I don't think the GP was inaccurate in his statement.
I agree with what you're saying, I just don't understand how it applies here.
Tons of people have created broken versions of popular encryption schemes. They go to Wikipedia, get the AES algorithm, and then implement it. That implementation turns out to be flawed, and by "rolling their own encryption" even if it is based on a very secure one, like AES, they have been incorrectly encrypting content.
But that doesn't apply here. They didn't create their own encryption scheme. Literally, nowhere in the code examples that I have seen did they reproduce either a popular or their own bespoke encryption routine (no maths == no encryption).
Concatenating strings together and then passing it to MD5() is authentication, but the actual hash function implementation is housed entirely within MD5() which presumably is created by someone who didn't screw it up.
People in this thread seem to want to entirely redefine what the term "encrypt" even means to encompass the entire authentication logic but either by the dictionary[0] or Wikipedia definitions[1] that is not correct.
If people want to create a new phase: "Don't roll your own authentication." I am absolutely fine with that. Just don't misappropiate an existing one.
"Don't roll your own authentication" seems to me to be even more pertinent advice than "Don't roll your own encryption" as it's a much more common mistake that I see at almost every single place I've worked at. I think it's not a common phrase because authentication is really not that hard and most engineers think they can do it properly. My experience, at least, shows that they can't. Everything from plaintext passwords to symmetric encryption happens and it happens at companies one would expect would implement some security. I've seen CTOs and Engineering Directors argue against changing the plaintext passwords till they were blue in the face and I just gave up. Obviously, our industry has done little to educate people on this issue and in almost every single major breach, it's easy to see how this was a key factor in making the leak worse by leaking the actual passwords or their improperly stored "hashes" (because in some cases they're not even hashes).
I guess I'm just use to hearing "crypto" used as a term that means both the applied part, as well as the math part, so whenever I see someone else say "don't roll your own crypto", I assume they mean everything from the primitives used all the way to how they used them. Sorry for the confusion
> Cobbling together existing primitives counts as "rolling your own crypto".
MD5() isn't a primitive, it is an entire implementation.
As I said, if they didn't do maths they didn't do encryption. You're conflating authentication with encryption, which aren't the same thing at all (just like an engine and a car aren't the same thing, or a heart and a human).
Encryption (and or hashing) is an important part of authentication, but the term "rolling your own encryption" clearly relates to the encryption-part of the authentication routine. People regularly roll their own authentication routines (even if that within its own right is a bad plan).
I am in no way defending that code. It is garbage. But people in this thread have clearly misunderstand the warning against "rolling your own encryption." Or don't know what encryption is.
Commonly used primitives
One-way hash function, sometimes also called
as one-way compression function—compute a reduced
hash value for a message (e.g., SHA-256)
Furthermore:
Cryptographic primitives are one of the building
block of every crypto system, e.g., TLS, SSL, SSH,
etc.
...
Combining cryptographic primitives to make a security
protocol is itself an entire specialization. Most
exploitable errors (i.e., insecurities in crypto
systems) are due not to design errors in the
primitives (assuming always that they were chosen
with care), but to the way they are used, i.e. bad
protocol design and buggy or not careful enough
implementation.
This last bit is usually what is meant by "don't roll your own crypto". Granted, the parent said "encryption", but he meant "crypto", and I thought that was quite clear from the context.
> This last bit is usually what is meant by "don't roll your own crypto". Granted, the parent said "encryption", but he meant "crypto", and I thought that was quite clear from the context.
That's how I read it — and therefore misquoted it — as well.
It is in the context of building a larger cryptographic protocol. But they aren't building a cryptographic protocol in this case, so none of that applies here at all, even a tiny bit.
In this case hashing functions are being used stand alone (both bcrypt and MD5), and therefore they're not cryptographic primitives. The only time they become cryptographic primitives is when they're used as such, in a cryptographic protocol, which it isn't here. You may have read that Wikipedia entry but I don't think you understood it, it says this quite clearly.
To be honest it is just flabbergasting we're even having this discussion. "Encryption" has a very specific meaning that is hundreds of years old, "crypto" is just a shorthand way of writing encryption or cryptographic, and neither term is a synonym for authentication.
I cannot tell if this confusion originates from people not understanding what the words "encryption" or "cryptographic" mean, or from them not understanding the difference between authentication as a broader topic and encryption/hashing as a singular component within it.
If people want to create the mantra of "don't roll your own authentication," then more power to you. But none of this has anything to do with "don't roll your own crypto." They categorically did not at any point roll any crypto, encryption, cryptographic protocols, or anything else related. Just custom authentication.
As I have said multiple times in this thread: "No maths used == no attempt at rolling their own crypto." It is as simple as that, so unless someone can point me to their bespoke implementation of crypto (i.e. maths) then what you're arguing doesn't make sense.
To use an analogy:
- Famous mantra is: "Never roll your own car engine."
- Article is posted where someone built a custom car using an off-the-shelf engine (even if an old and dangerous one (i.e. MD5)) and it crashes.
- Someone replies: "This is why you NEVER roll your own car engine."
- I reply: "They didn't roll their own car engine, it was a standard off-the-shelf one!"
- Someone replies: "The term 'car engine' (as in "Never roll your own car engine") now refers to the ENTIRE CAR, not just the engine."
- I reply: "No the engine is a specific component in the car." (i.e. MD5 is a specific off-the-shelf component in an authentication scheme).
- Someone replies: "The term 'car engine' now means 'car.'" (i.e. the term "crypto" now refers to the entire authentication process for some reason).
As I said I feel like slamming my head on the table. I don't know another way of explaining this. The fact you think that Wikipedia article has any relation to this topic at all means you're even further from understanding this than I thought. Do you really think an authentication scheme is a cryptographic protocol? Is AES an authentication scheme? Is Kerberos a cryptographic protocol? No, and no. Kerberos USES encryption and AES is used IN authentication schemes, but they're just components in both cases.
In the context of the quote, "crypto" means "cryptographic system", which encompasses authentication.
They tried to implement a "secure login token" by leaking the user credentials in quasi-plain text (with the advent of rainbow tables then GPUs, MD5 is barely better than Rot13 nowadays).
But still, there's math in that. Strings form a monoid under concatenation and toLowerCase() is a pure function (probably 'map (_ $ 223) source' for a given subset of the domain).
To keep with your (insulting) analogy, they tried to build a car by pouring mayonaise on a crankshaft when they really needed a trumpet.
By doing so, they ruined the protection offered by their otherwise strong password hash.
Don't roll your own crypto. Also, assuming your interlocutor has a modicum of intelligence is basic courtesy.
> It is in the context of building a larger
> cryptographic protocol. But they aren't
> building a cryptographic protocol in this
> case, so none of that applies here at all,
> even a tiny bit.
"Crypto" in the context of "don't roll your own crypto" does not mean "cryptographic protocol", it means "crypto system". The Wikipedia page uses protocols as examples of crypto systems.
> "Encryption" has a very specific meaning that is
> hundreds of years old, "crypto" is just a shorthand
> way of writing encryption
You're wrong. As I said before, in the case of "rolling one's own crypto" it refers to "crypto systems". Not every word containing "crypto" refers strictly to encryption. E.g., MD5 is a "cryptographic hash function".
> or cryptographic, and neither term is a synonym for
> authentication.
> ...
> I cannot tell if this confusion originates from
> people not understanding what the words "encryption"
> or "cryptographic" mean, or from them not
> understanding the difference between authentication
> as a broader topic and encryption/hashing as a
> singular component within it.
Cryptography encompasses more than just encryption. Authentication is included in the definition of "cryptography" here: https://en.wikipedia.org/wiki/Cryptography. An authentication system is totally a crypto system and totally falls under the definition of "cryptography"!
MD5 is a bunch of XORs and MODs. Bcrypt has a bit of that, but mostly it uses Blowfish. Scrypt is used for more or less the same purposes as bcrypt, and is built on top of PBKDF2_HMAC_SHA256. Do you see the difference? Bcrypt and scrypt are built out of things at MD5's level.
Typically, a cryptosystem consists of three algorithms:
one for key generation, one for encryption, and one for
decryption.
That sounds like bcrypt and scrypt qualify. I don't know, but they are clearly much closer to it than MD5. Anyway, the distinction is not always black and white. The point is to use as high-level and complete an implementation you can find for any crypto-related things you need to do. Including password hashing. MD5 was too low-level, and look at how badly they messed it up. They tried to achieve what bcrypt and scrypt were designed for by using a much lower-level construct in MD5. They rolled their own crypto.
> Do you really think an authentication scheme
> is a cryptographic protocol? Is AES an authentication
> scheme? Is Kerberos a cryptographic protocol? No
No I don't. But an authentication scheme is a crypto system. Kerberos is a crypto system. Hence the term "don't roll your own crypto" would apply to both.
MD5 is a primative. Many algorithms specify a hash/compression function (which MD5 is) or a permutation (which AES is) in how they work. Message authentication (HMAC) can use any compression function, key derivation can use any compression function, and so on. Even stream ciphers can use MD5.
People implementing things that use crypto should only be using functions that are "Authenticated Encryption" which specify the full suite like AES-GCM (permutation function: AES, in Galios/Counter Mode), or "Key Derivation Function" (PBKDF2-HMAC-SHA2 or Argon2d-Blake2b). What they've done at AM is implement their own key derivation function using the MD5 primative and concatenation.
If you want to do crypto, you have to do crypto. You can't just code up an algorithm one weekend and use it for the rest of your life (or the rest of your business's short life). You have to code up a bunch of algorithms, read a bunch of other people's implementations, learn the strengths and weaknesses of different techniques, follow the trade literature, go to conferences, argue with other cryptographers over beers. Be plugged into the sorts of networks which will let you know when your favorite crypto technique is starting to become weak.
It's fine to have a diverse set of interests, try your hand at various things. But some things you have to really commit to. You can get away with half-assing a lot of things, but some things -- like cryptography -- need to be fully-assed.
There's a US Govt agency called NIST. They sponsor "contests" basically for academics and engineers to put forth proposals for crypto algorithms, which are then reviewed. I'd read up on here[1], if it interests you at all, you can find the original academic papers regarding each of what became the standards, and I think they have references as to the review process as well.
There are also contests run by groups without any NSA affiliation, such as the CAESAR contest for authenticated encryption [1] and, relevant to this subject, the Password Hashing Competition [2], which recently decided on a pretty impressive winner.
Yup, groups of expert mathematicians, also known as "cryptographers". They create and review on another's schemes.
The author of a scheme can be a single competent person, but it would be unwise to trust it before peer review, regardless of the author's track record.
They did actually change the title [1] - It was misleading before, and in fact the heading of the article still is - "How we cracked millions of Ashley Madison bcrypt hashes efficiently" - impliying they are able to crack bcrypt efficiently (they can't).
This should be extended to "Never roll your own security", because that's really the crux. As many, many people have pointed out in the past, that's why books like Modern Cryptography exist. In these books, they basically say "hey, use these functions and these parameters, and you probably wont die", but in this case, ALM said "screw those parameters, we're smarter than you!!!!" and rolled their own security. As noted by the article, bad move.
I understand the meme: "Never ever roll your own encryption", but I am stubborn enough to ask: "Why?".
Why use bcrypt over scrypt over pbkdf2? Because crypto experts told you? How could you know it is correct, if you are not able to inspect it? By the amount of people yelling "Use bcrypt"?
I know that in crypto you should expect an attacker to be able to read your source code. If you can keep secrets, while your source is out in the open, then it is good crypto. But does that mean you do not need a script-based salt? An attacker which can get into your database, should be able to get code-read access too right? I don't think so... Databases are leaked on forums without any trace of the source code/app logic. When these people did not roll their own encryption, any attack which is able to beat modern crypto (you will never hear of this, as you are not an expert), could now attack you. They fingerprint the hashes, try to find out which expert roll you used, open their suitcase of crypto breaking tools written by the same expert when she was working for the NSA, under cover of doing a PhD at MIT, and go to town.
Don't listen to me, because I am not an crypto authority, but do roll your own encryption: Give your own twist to it. That is security by obfuscation, and would not put all eggs in the same basket. An attacker has to be able to break your custom scheme now, for every different site/database attacked. It could be simple, it could be near perfect, but it won't be as simple as pressing a button on the "break modern crypto"-toolkits.
If you are one of the few doing this: People will move to less arcane targets in the never-roll-your-own-basket. If you are one of the many doing this, breaking crypto would become an unmanageable field of eggs.
If I was a state actor in charge of keeping secrets and breaking crypto, these two memes: "Never roll your own encryption" and "just use bcrypt" are exactly the memes I would propagate to the tech crowd. Even moreso when you can already break bcrypt (or expect to in 5 years and just store everything that looks encrypted with bcrypt) and want to keep your task manageable.
AM would be harder to crack if they'd ROT-13'd the hashes in the source code.
When you do that you end up losing entropy or leaking information. Just because you have no idea what you are doing doesn't mean than it won't be trivial for someone else to recognize and exploit.
The amount of people telling you to use bcrypt has nothing to do with it. It's the peer review conducted by hundreds of experts that understand information theory that is the indicator. Crypto experts aren't just randomly shifting around bytes and hoping it works, modern protocols all protect against various attacks that you are going to expose yourself to by ignoring them.
Even if you're not an expert, you will immediately hear of any attack on modern crypto because it will be a huge deal. These are algorithms the NSA recommends to other arms of the US government that they are protecting.
If the attack is not made public, you will be screwed anyway if you are a target because all of your OS update mechanisms (package signing, etc) all depend on modern crypto so an attacker with the ability to break that will see your super secret hash function of "count the 1s" anyway.
There are well documented reasons to use bcrypt/scrypt/etc over things like MD5/SHA1/SHA2. It's mainly a problem of hashing speed. It is also not impossible to understand how these algorithms work (and understand why they are more safe/take more time). If your password hashes are dumped, it's a question of time before they're decrypted. Depending on the algorithm you use, that time can either be minutes/hours, or it can be days/months/an infeasible amount of time.
You are correct that the implicit chain of trust around why you should use those things should not be free of suspicion, but that is a terrible reason to not use state-of-the-art techniques.
Modern crypto is demonstrably hard to crack because of mathematics. The question of whether it is all broken is there, but it's much harder to break/cheat mathematics than anything else (and again, proofs exist to prove stuff).
DO NOT build your own encryption, or put your own "twist" on any existing well-known methods. What you think is clever might take an attacker 10 minutes to figure out. Take a small pill of humility, you're not as smart or original as you think you are.
There is no "break modern crypto" toolkit. Most toolkits that script kiddies use are around broken APPLICATION of security intense. Assuming RSA/AES are not broken, then only theorized attacks require quantum computers. In 2015, it is highly unlikely that your adversary will have quantum computers, unless they are the NSA, and then your problems are much bigger than that (ex. if you interact with any company in the US, you are hosed). The overwhelming majority of businesses are compromised from things like phishing or running (discoverably) outdated software on their servers (ex. Some super old version of tomcat with known vulnerabilities, that announces itself in the HTTP header).
If you have information crackers want, your little security scheme will get owned. It is better to put your trust in proven/provable mathematics, even if you are not an expert. Arguably, your adversary is the kind of person that ENJOYS solving puzzles. Adding one more puzzle is not going to turn them away, it's going to make it even more fun.
When you have a sufficiently bad injury/infection, you don't go try and work up your own remedy, you go to a doctor. The fact that you didn't go to medical school and may not necessarily trust your doctor doesn't make it a good idea to start making up remedies for issues that have been well-studied by others.
I recently found out that piwik also uses a login token of the MD5 of the password[0]. So this mistake is still very prevalent.
If you want to provide a one-click automatic login to Piwik for your users, you can use the ‘logme’ mechanism, and pass their login & the md5 string of their password in the URL parameters:
That's actually a whole lot worse than the AM version here. The MD5 hashes themselves are usable as valid password!
And the cracking of the MD5 hash back to original password is fully amenable to rainbow tables. All you need is hashes which you can extract from DB or webserver logs... Or sslstrip'd/HTTP traffic if that's possible.
they stored a static login key generated by md5(strtolower($username).'::'.strtolower($password)); - so they could crack the md5 part easly and bypass the bcrypt encryption
Slightly pedantic: "Discovery 1" email indicates that the $loginkey encryption was the weak md5 method until it was changed to bcrypt in a 2012-06-14 commit.
As a result, any accounts that were created before 2012-06-24 and that did not have their password changed after that date (which would generate a new bcrypted $loginkey) were vulnerable.
Completely agree, Match.com does the same thing. Not so long ago a user signed up to their site using my email address (never figured out why).
They were able to create an account and subscribe to the site without ever verifying the email, so for a week or so I was getting notifications sent to me without any way to unsubscribe from the email.
Clicking any of the links in the email signed me in as the user and gave me full access to their account and billing information. I ended up going into their account and turning off all email notifications to make the emails stop.
Edit: Just checked my trash folder and an email sent on the 8th of August still contained valid login keys to access the account.
> Not so long ago a user signed up to their site using my email address (never figured out why).
If they used legit billing information, with a name other than yours, it was probably a legitimate typo of the email address while signing up, using copy-paste to avoid typing the email address a second time. They probably haven't yet realized they aren't getting emails for their account.
Someone (or two people with the same name) has signed up for a Hertz car rental account and a trash removal service account using a name that could reasonably be mapped to my gmail username. I presume they either have <my_username><2/year/other_number>@gmail.com or <my_username>@<some_other_provider>.
You'd be surprised how often this happens. I had a similar situation with someone who accidentally used my email when buying a new car.
For a while I was getting emails from the Hyundai dealership that had auto-login links that would have let me do all kinds of things, including requesting a (paid) tow of the car from my house back to the dealership, scheduling (or cancelling) maintenance, ordering extras and part, and more..
Luckily through that logged-in area I was able to find the individual's phone number and we texted back and forth until he understood the problem and called his dealership to update his info.
This is genuinely amazing to me. And the IT guy that set up that Hyundai system is probably getting paid plenty to do it, despite massive flaws like this.
Security is a cost center with no output and no drawbacks when you cut it (as the business people see it). So it is often quickly underfunded til some major incident.
That's really strange, isn't it? Because it only takes one major incident for a company to go entirely out of business in many cases. Or for CEOs to get ousted or step down, as in the Ashley case, or any number of other irreversible catastrophes. It would seem to me security should be the most important part of the entire process.
Then the CEO blames the IT dept, shareholders lose their investment and CEO moves on to next job with the bonuses he got for cost cutting still intact.
I find it hard to believe that they didn't know better. They were using 12 rounds of bcrypt to store the passwords, afterall. Assuming just the DB is compromised, they would likely be okay because of that. I'm not sure how often systems are so entirely pwned as to gain access to their source code repos and their entire production database. That seems like an outlier kind of attack, though I think the same thing happened with Sony.
I don't think it's fair to say that this was a security oversight, so much as it was a conscience decision to make the system have less friction for users by utilizing these login tokens.
I have a nice Volvo dealership in California that is sending me updates on someones car service, as well as the billing for some storage locker as well.
Apart from that, i get about 5 to 10 of real emails like this each month. Looking at the email addresses in Gmail i can see a lot of address that try to use a period in the address.
What's the risk of using plaintext passwords if we assume every user is employing long, random, unique passwords? This has always seemed like a non-issue to me because I've been using a password manager for a half-decade.
e: Downvoting questions is mean. FWIW I always use bcrypt.
What's the risk of using plaintext passwords if we assume every user is employing long, random, unique passwords?
This assumption is known to be false. We should not design systems around this assumption. If we do, popping a random startup will give up Gmail accounts, by the thousands to tens of thousands, and since we've settled on email accounts as The One Ring To Your Identity that imperils their brokerages, domain registrars, employers, World of Warcraft characters, physical safety, privacy, cat photos, etc.
I don't quite follow. If they have access to the server's database, then unless the rest of the DB was encrypted using a key derived from the users password, will they not have access to all the accounts anyway?
(edit: I should perhaps make it clear that I'm not suggesting that passwords shouldn't be hashed, merely that if you have access to the database containing the password hash, then in many cases you'd already have access to the data that knowing the password behind the hash would provide).
If we assume every user is employing long, random, unique passwords ... then we surrender the ability to reason about the real world. Economists are fond of that kind of thing, but in IT security it's not terribly helpful.
Many (most?) users employ the same password on many different sites. So now an attacker has email/password pairs that work not just on the site that was compromised, but also on Gmail, Facebook, Twitter, Amazon, banking sites, and so on.
The login process looks like: User navigates to login screen and sends plaintext password to server. Server hashes the plaintext password and compares it to the hash on disk. If the hashes are equal, the server grants access.
Since a user can only submit a plaintext password, not a hash, then they won't be able to log in unless they know the correct plaintext password. Even if they have a copy of the server's db, they won't be able to use any info within it to log in as anyone else unless they use brute force to reverse a hash.
EDIT: It's true that the attacker might have a copy of all the user's data. But unless they know someone's plaintext password, they can't log in as them. The attacker might have a copy of your tweets, but they won't be able to log in as you and send a new one.
There are only a few ways an attacker can snag a database. Most of the attacks let an attacker download it, but not modify it. For example, directory traversal lets you download ../../../../etc/passwd along with ../../path/to/database_backup.
Even if an attacker has shell access, it's unlikely they'd be able to use the service or change a user's password by manipulating the database directly. They might not be able to modify database for a number of reasons: the database might require credentials the attacker didn't get, or the attacker might only have access to a server containing database dumps rather than the database itself, or the attacker might be logging in as a user which has read-only access to the database file.
I get that - but in a majority of cases, isn't "juicy" information going to be held in the DB anyway? Sure, you won't be able to go in through the "front door", but if you can get a copy of the password hash from a database dump is it not pretty much game-over anyway?
If you're able to perform SQL injection against the DB, would you not be able to change the password in order to gain access to the system, or grant admin privileges to your own accoun
You can get a password hash from other sources besides a full DB dump. SQL injection, poorly generated tokens in URLs or cookies, etc.
If you're able to perform SQL injection against the DB, would you not be able to change the password in order to gain access to the system, or grant admin privileges to your own account
Often it's easier to do an injection against a SELECT, just forcing it to read an extra field, particularly now with ORMs that won't pass straight SQL through.
Sure - in this specific case I was referring to "any attacker who snags the server's database" - I was taking snags the server's database to mean "has full DB dump", which perhaps was not sillysaurus3's intention.
I agree that if the only information you have is a users password hash then your options beyond attempting to brute-force the hash are limited (unless the system was vulnerable to a pass-the-hash attack, of course).
If an attacker somehow downloaded all my databases (because we keep data partitioned to minimize damage if one of them is compromised, as well as to better partition the application itself), then they would be able to see all of history.
But they wouldn't have the user's credit card, because that's encrypted separately with the key stored somewhere else.
And they wouldn't be able to take action on behalf of the user. That is, they couldn't place new orders as the user, or stuff like that.
It depends what you're after. If you see that someone is subscribed to Ashley Madison and what to blackmail him/her, then you have what you need. On the other hand, the password is the most important thing, because the majority of people use the same password everywhere; if you have the password from AM, maybe you have the one for GMail or for the bank account, where the real "juice" is at.
> if you can get a copy of the password hash from a database dump is it not pretty much game-over anyway?
Only if you are using a weak hash like MD5. The primary point of hashing a user's password is to make so that it cannot be easily cracked in the event of a DB breach/leak. Stronger hashes require much more time, energy and computing power to crack...far more than what is realistically available today.
> If you're able to perform SQL injection against the DB, would you not be able to change the password in order to gain access to the system, or grant admin privileges to your own accoun
Depends. You would need to know what hashing algos/processes, initialization vectors, etc that are in place to create the DB hashes. For example, if they are "salting" the passwords, you would also need to have the salt token on hand.
> Only if you are using a weak hash like MD5. The primary point of hashing a user's password is to make so that it cannot be easily cracked in the event of a DB breach/leak. Stronger hashes require much more time, energy and computing power to crack...far more than what is realistically available today.
I'm not suggesting that they'd be able to trivially crack the password (nor am I arguing that password hashing shouldn't be a thing - it certainly should be). I'm saying that it's likely that if they can access the database that contains the user's account details, they're able to access other information that's held in the system anyway (that's what I meant by "game-over").
> Depends. You would need to know what hashing algos/processes, initialization vectors, etc that are in place to create the DB hashes. For example, if they are "salting" the passwords, you would also need to have the salt token on hand.
I bet in a lot of cases (not all, would require open-signup etc), you could sign up for an account on the service using a known password, dump the hash of the known password, and swap out the hash of the privileged user. That way you wouldn't need to know the details of the hashing implementation.
Hopefully the important data is also not stored on the system in clear text and the key to decrypt it is stored on a different server only held in a secure area of memory while the process is working.
Password have value beyond one system they can be used to impersonate the user and often they are reused on other sites like email accounts (which can be used with reset password almost everywhere to gain access to those sites).
"you could sign up for an account on the service using a known password, dump the hash of the known password, and swap out the hash of the privileged user. That way you wouldn't need to know the details of the hashing implementation."
The hash should also have a salt included (which hopefully is also not stored on the same server as the database) so hashing is salt + password =hashingfunction> hash. The salt can also be a supplemented by a user specific info (unique account id) so that hashes for one user wont' work for another
Perhaps, but it would then depend on user privileges set in the database. If the application is accessing the DB as root or as a user with full admin privileges, then yes, I would say that would be a "game over" situation. On the other hand, if the DB user that is being used in the injection attack has read access to a limited set of tables, the damage may be much less severe.
> and swap out the hash of the privileged user.
True, if the initialization vectors of the hash are the same for admin and regular user accounts (for example, using the same salt for admin and normal users). You would also need write access to the DB via your injection attack.
This is, I think, actually a very reasonable argument. The threat model which compels us to use specific strong techniques to hash passwords is very much a last line of defense: If an adversary has got hold of a copy of our database, and the user has chosen a relatively weak password that's vulnerable to rainbow tables, and they have reused that password on their banking service, then the fact that their password is bcrypted rather than MD5'd might prevent their bank account getting hacked. Bcrypting all your users' passwords is a just-in-case-the-worst-happens measure, like issuing a cyanide capsule to a spy - "Alright, you've captured me, but you'll never get any information out of me!"
But it should be clear that a lot of other stuff has to have gone wrong before it comes to that, and - more importantly, why are we all of a sudden so concerned about whether the user's bank account will get hacked? That's the user's problem, not ours, right? If someone's stolen our entire database, things have already got as bad as they can for us - if they're also bad for our users, why do we particularly care?
Fundamentally, if a user has elected to give us their bank password, why is it on us to protect it with military-grade crypto, when clearly the user doesn't think it's sufficiently valuable that they kept it secret? And why should we act as if every password someone chooses to use might be a shared credential?
It seems you're expected to assume that it's your duty to protect customers against any loss they may incur on a third party site because they reused a password, if that password is revealed by someone hacking your database.
But total DB compromise isn't actually the only place in your threat model where you should be worrying about who can figure out a user's password. One very good reason for strongly hashing passwords is to make it very clear that your employees who have legitimate access to the database can not themselves discover users' passwords. This helps maintain the trail of accountability that lets you demonstrate that actions taken by a specific user who logged in with their password were actually taken by that user.
In that sense, the real reason for using strong crypto on user-chosen passwords is because they are secrets which belong to the user and which nobody else should be able to find out about. As such we take extraordinary measures to ensure that that secret is kept. That way it doesn't matter if their secret is that they use a weak, nonunique password. Indeed, how could you even know?
In the Ashley Madison case, of course, it turns out that there are a lot of other secrets that a database can be holding on behalf of a user that should not ever be accessible to anyone else.... but nobody's complaining that the activity history of accounts wasn't encrypted to protect customers against loss after a total data breach.
They would have read access to all of the database. However, if they also have the password, then they have they ability to impersonate that user and take actions as well.
As an example, if you're a bank, if someone gets your database of data, but the passwords are well protected, they can find out how much money you have in your account. If they get your password, too, then they can login as you and transfer your money to their offshore account.
They may not necessarily have compromised the entire system.
Consider for example, SQL injection. Alternatively, some kind of DB dump mistakenly exposed in the webroot. Neither should happen but they're real-world examples. Sometimes misconfiguration leaves you in a weird state.
Long, random password stored in plaintext: as soon as the user database is compromised, the attacker can log into your account and do whatever they want. This may result in additional exposure of information that wasn't part of the original compromise. The attacker can also impersonate you and cause other kinds of damage.
Long, random password hashed with bcrypt/scrypt/etc: if the user database is compromised, the attacker gets the user database and nothing else.
Chances are if they have database access, they can do that already. Many/most exploits at the application layer are made to gain access to the database.
There are a significant number of ways paswords/password hashes stored in a DB might be revealed that wouldn't result in write access to the DB.
Off the top of my head:
(1) Timing attacks against string comparison (in the hash cases, this usually also implies a guessable salt)
(2) Misconfigured servers showing exceptions/debug info to users, coupled with sensitive information in debug messages, coupled with a remotely triggerable exception
(3) Compromise only of the offsite backup site containing DB dumps but no access to the live DB
(4) Flaky drive is replaced without being physically destroyed. Dumpster diver gets drive working again briefly and gets partial/all DB contents at drive failure time.
(5) Someone else compromises the DB and posts a dump on the web.
A big part of good security is taking steps to keep small errors from being leveraged into big problems. (Though, I'm not saying that read-only access is only a little problem.)
Account read-write access opens up a lot more opportunities for fraud and abuse vs. account read-only access.
Your comment is killing the layout of this page. Can you please fix the formatting of your message? HN does not support arbitrary markdown. And even if it did indenting by four spaces would yield pre-formatting not an ordered list.
SQL Injection is an application layer exploit that gains the attacker access to the database, but does not involve any sort of privilege escalation. Consider this: an attack gain access to your bank's user credentials database. Do you want them to have access to your unencrypted password? Is it really so "harmless"?
If I breach your environment and get access to bcrypt hashes, they are useless to me as-is.
If I have enough processing power to brute-force compare them, I can eventually get the original password back, but that isn't a zero-cost effort in terms of time. As soon as you realise you have been breached, I would expect you would have initiated a lockdown of some kind, either preventing access to accounts until the owner can confirm identity, or forcing a password reset for all users.
If you store passwords in plaintext, I can copy the passwords, and impersonate users even while they continue using their accounts, unknown to them.
No. Given a large enough work factor, there literally aren't enough resources in the entire universe to brute-force even a single hash. "Hard to imagine" and "limitless" are very different animals.
Edit: moreover, the GP has already mentioned that "[brute force] isn't a zero-cost effort in terms of time," even if you have access to humongous amounts of processing power. The point of encryption isn't to protect data forever; just long enough for them to become uninteresting.
> Given a large enough work factor, there literally aren't enough resources in the entire universe to brute-force even a single hash.
Or, given a strong-enough original password. I postulate that, until the moment I posted it, '3BgZhZcGPcHswmfic79jgjsfqyMKW1Rh1rkFjagJC090V2tljd9Zy2vpYJw944HG' would never have been brute-forced, even with an extremely low work factor (in fact, it wouldn't have been brute-forced even without bcrypt, scrypt or PBKDF2: even were it just stored as a simple SHA2-512(pw) (i.e., baafbe64d25f27d4b7b2965f9abf43cf25acd8fc6021798a90253f33b3071e9
fd145eaccd555290913c8679fd4acf949f05243ac1089548abbab0e0ed77e63
31), no computer on earth, nor all the computers which ever have been or ever will be, would have brute-forced it (unless there's something we don't know about SHA2, or about computing).
Lesson? Always use high-quality random passwords. For a 128-bit security level, with upper- and lower-case letters & digits (62 characters, which means each character yields (log 62 2) = 5.9542 bits of entropy), you want a 22-character truly random password. 'xNl5gkbgXntyxi3oNO1dML' is a perfectly fine password (save for the fact that it's now public).
According to the NSA [1], 256-bit keys should be used to protect data up to TOP SECRET; that works out to 43 characters. Hence, 'TIdmzj1COc9ECwcilyQYVfLl5Dt8ti3UUlDVULrUrfx' is as secure an example password as could be.
(Yes, I'm assuming here that your password-generation algorithm is truly random. If not, do not pass Go; do not collect $200…)
While uncommon, it's not ridiculous. A site could, for example, decide to generate a user's password rather than allow them to provide their own, thus guaranteeing it be long, random, and unique. However, it's still not a good idea to store such passwords in plaintext.
I'll second nilved's suggestion of KeePass. The database is encrypted and stored on the local machine. I currently use Syncthing to share it between my devices.
Likewise, I'm very happy with this exact setup after coming from a mix of memorized password and site-dependent password-generation schemes.
I'm on Mac and found KeePassX to be a better solution than the original KeePass, it's much lighter weight. My only hope is that KeePassX gets browser integration at some point via keepasshttp - https://www.keepassx.org/dev/issues/91
From what I've heard, people are generally just concerned that the database is not authenticated whatsoever, and so a malicious actor can modify your database (particularly if it's hosted on Dropbox/etc...).
This doesn't seem like a particularly scary vulnerability, but one scenario that was brought up (though I'm not 100% convinced), is that a malicious actor can corrupt the password for one website, leading you to change the password.
I dunno, don't see it as a huge vulnerability, but I do agree there is little reason for such a database not to be authenticated.
that is a ridiculous assumption that only leads to really silly "if the 8 planets are aligned just so" kind of justifications that make people feel fine with awful security practices.
Like protecting your business with an industrial grade door locks on a building made of hay. Just a whole lot of cheating going on over there, ouch.
edit: I don't know if this came up before, but based on how they stupidly tried to cache the login session tokens with md5, instead of running through the 12 work factor bcrypt, I can assume that they saw this as a bottleneck.
Instead of dropping the work factor or doing this caching baloney, could a service be made that runs on extravagantly fast hardware, which provides an API for strong, high work factor bcrypt, pbkdf2 based authentication.
I can assume that at around 10 rounds, each attempt takes about 50 - 100 millis
The "remember me" token doesn't need to contain or be derived from any meaningful data at all; it merely needs to be unique to the user so you can associate it with the user, and should be both unpredictable and frequently changed/regenerated. It's just a more persistent version of the session ID you'd be using in any case even if the "remember me" option wasn't selected.
One point is not clear to me: did the crackers know $username's already, or did they perform some kind of dictionary attack? Brute forcing both $username and $password out of millions of hashes seems a bit hard - even considering md5 trivial, not employing an hmac scheme.
They have the db dumps, so yes, they know the usernames. And they used a rainbow table[1] to break the md5 hashes, which is a lot cheaper than brute-forcing.
Then would replacing md5() with hash_hmac('sha256' [= or whatever ], strtolower($username) [= data ], strtolower($password) [= key ] ) help against such an attack? If I understand correctly, this would have ruled out rainbow tables.
Edit: clarified. BTW hash_hmac is built-in with PHP >= 5.1.2.
The problem isn't just rainbow tables. Md5 is a fast algorithm. Sha256 isn't much slower than md5. You want anything auth related hashed using a slow algorithm - speed is your enemy.
That's why bcrypt and the like are popular, because you can adjust how long it will take to calculate.
AM used an insecure login token at one point, and 3 years ago they fixed it. They switched from an MD5 of lower(pass)+username to an MD5 of the bcrypted pass+username, which is no longer reversible.
Apparently they never updated all of the previous login tokens though, so anyone who had created an account before the new secure system was put in place still had a vulnerable token stored.
When it comes to security, when you fix something - fix it for everyone people! Even if it's hard.
The good news for these folks is that the passwords revealed appear to be over 3 years old, and we all chang our passwords more often than that, right????