This is based on NaCL. NaCL is an extremely great library that we virtually never recommend because it is very difficult to integrate into most people's dev environments.
Libraries like NaCL, Keyczar, and Cryptlib work by removing all the design choices from cryptography. You don't pick the key sizes, you don't pick the algorithms, you don't even pick what kind of keys you exchange. They implement a whole cryptosystem, as if for some new unreleased version of PGP, subtract the file format, and present it as an API. They're great. They are literally the only way you should be deploying cryptography in your applications.
I think this is a great development, but I am not qualified to say. This sits in a weird place in between a language binding for NaCL and a fork of NaCL. As a bindings package it's an overachiever; as a fork, it seems to have made extraordinarily conservative changes.
You'd really like a real crypto dev --- Matthew Green, Colin Percival, DJB (NaCL's primary author; @hashbreaker on Twitter), Steve Weis, Trevor Perrin, &c --- to say "this is all totally sane, and if you buy the premise of NaCL, go ahead and use this instead.
You'd also really like Sodium to say "this is as much as we're changing from NaCL and nothing else", because what makes NaCL worth building on is the expertise that went into designing it in the first place, which, like me, Frank Dennis probably doesn't have.
This looks like new development that's just API-compatible with NaCL though, right? Which feels a little strange.
The reason I'm drawn to the work of DJB is because of his fanatical attention to security-related detail. He wrote and released his own high performance mail server, DNS server, web server, and logging replacement during a period of time when server-side software was plagued with exploitation, and people have only ever found, what, two somewhat minor bugs? Ever?
So what he creates is almost always amazingly solid, but for all his attention to security perfection, his software is also amazingly unusable. Want to run publicfile? First, forget everything you know about standard unix directory structures, because DBJ doesn't like them.
I feel somewhat the same way about NaCL. We can bet the lives of our first born children that the implementation won't suffer any of the bugs that OpenSSL has over the years. Side channel attacks, memory corruption bugs, and timing problems will almost certainly be nonexistent, and someone could probably do their PhD thesis on documenting why.
But like his other software, NaCL has weird usability quirks. This project points out that some of those quirks were certainly manifest in the implementation, but some of them shine through the API itself as well. Like his other software, my sense has been that interacting with NaCL is something that we would almost have to put up with in order to avail ourselves of DJB's greatness, rather than simple joy.
So a rewrite that's API compatible isn't super immediately appealing to me, in the same way that I don't think I'd be particularly drawn to an interface-compatible version of qmail or publicfile that was written by someone else, when the value was in DJB having written them to begin with rather than the interface (the horror!) they provided.
Although to be fair, it's certainly not as if anything else is that much more usable in this world right now.
As discussed on the mailing-list, there will be a high(er) level API, similar to the C++ API, that doesn't require dealing with input/output being placed at different offsets in a shared buffer.
This is not a rewrite. It uses the reference implementations from NaCl and the same constructions. What it brings over NaCl is a standard build system.
Just a note on the approach taken here: I can't say djb would approve of Sodium, but when I asked him about NaCl's portability, he had this to say:
"The real work here is making everything PIC. Of course, if what matters is the API rather than speed, then achieving PIC is easy: just remove the asm."
Sodium is mostly a "just remove the asm" project, so it seems like djb doesn't hate the concept at least.
I thought he had implemented it in terms of NaCL (I hadn't bothered to look at the code, because I worried I'd decide I was smart enough to endorse it).
Judging from the context below and my limited knowledge of NaCL, this is incorrect. It uses the same algorithms but with the ASM removed. Since, I believe, one of the points of the ASM was removing timing attacks, removing it is not an inherently safe operation. Not saying anyone did anything wrong, just people should be careful and someone should probably check. This is not just a wrapper around NaCL.
Everything that doesn't have assembler versions of all their constant time code (even NaCl under x86/x64) will be vulnerable then. It's possible that compilers start to "optimize" the branchless constructs, but it would almost be malicious as they extremely rare to ever encounter outside of specifically wanting not to branch.
While the people you've mentioned haven't been keeping an eye on it, Jean-Philippe Aumasson has. There have been a lot of other "mid-level" crypto people who have been keeping an eye on it as well and giving feedback to Frank. For example, at one point Frank switched out the default random number generator to use XSalsa20 instead of /dev/urandom. When this was pointed out, Frank addressed it by switching back to /dev/urandom (and a separate codepath for Windows)
Note: I've talked to Matt Green about Sodium. About all he had to say was as soon as you modify NaCl, you've violated the djb warranty ;)
In what way is NaCL difficult to integrate into existing dev environments? I've not personally used it before, but I've been looking for a library that abstracts away some of the difficulty of cryptography (granted you can't hide it all) and NaCL appears to label itself as such.
Are you just repeating words you found elsewhere in the thread? You'll be hard pressed to find a crypto system that doesn't require implementations at both ends. NaCL is the library used to implement DNSCrypt, which is pretty much ssh for your DNS queries.
I don't see anything in a Github repo to support this, not with the Microsoft tools. Perhaps it builds under Cygwin and similar, but that's hardly qualifies as "builds on Windows" for practical purposes.
Thanks, I see. Any plans to add static lib generation for Microsoft tools (an nmake script) or is there some gcc-isms in the code? DLLs aren't exactly the most desirable arrangement in a heck of a lot of cases.
I just completed a major cryptographic project overhauling the security at my company. (Re-doing our password hashing and authentication, and the protocol for all our networking and communications.)
The password hashing was easy: a portable implementation built as a static / shared C library + language bindings linking to OpenSSL's `libcrpyto` for PBKDF2 (with an alternate implementation using CommonCrypto on iOS / OSX).
The encryption and authentication layer for the communications was much tricker. The first draft was an implementation based on industry standards: RSA2048 + AES256. It needed to be portable to iOS in addition the various other platforms supported, and Apple has deprecated OpenSSL on iOS and OSX. Annoyingly, OpenSSL does not ship with darwin-arm support out of the box, so a custom compile was not an easy option either.
In the end, I ended up picking NaCl, and specifically `libsodium` as a portable implementation. The library, unlike OpenSSL, is beautifully designed and very easy to use, and implements asymmetric crypto functions (based on elliptic curves) which are actually much superior to RSA, providing much greater security for much shorter key length.
Btw: the CocoaPod for libsodium is broken. I had two problems: 1) the headers include path for the pod was not set correctly, and 2) one of the algorithm's alternate implementations would not compile with clang. Removing it and using the ref implementation instead did the trick though.
I'm just lamenting to no one in particular that in the universe of available words/acronyms/abbreviations to describe projects we have as many collisions as we do. If I had to choose (ugh), DJB's NaCL acronym is at least more specific to its purpose than NaCl meaning "Native Client" just to make a salt + pepper pun. I guess all I'm saying is that if I ever write an open-source slab allocator, I won't call it OpenSSl.
If it works as suggested, this library binding will be a godsend to frontline developers. Cryptography is exceedingly hard to get right and unfortunately existing libraries often tend to be written for developers who are already familiar with cryptography. Hopefully this will see the adoption it deserves and we'll see less incidents like the Mega encryption scandal.
that's true, but it's the highest-profile encryption drama I could think of - a more appropriate example would probably be the onslaught of companies which are storing passwords in plaintext or simple MD5 hashes.
Hate to keep doing this to you† but you don't need anything like NaCL to solve that problem; just use bcrypt. Every development environment has a bcrypt.
Good example of a flaw mitigated by NaCL: the CBC padding oracle attack.
keyczar is about to support python 3 (there's a patch) and i was planning to make simple-crypt delegate to keyczar (or just delete the project entirely, since its only reason for existence was nothing better existed on python 3, but people seem to be using it). should i delegate to this instead of keyczar? what is the difference?
Keyczar has support for Java, Python, and C++ and uses the usual suspect standardized algorithms: AES, RSA, DSA, HMAC-SHA1. Google has been pretty active in maintaining Keyczar lately. ECC support should be added at some point.
As bascule said, NaCl / Sodium are using DJB's suite of algorithms, which are going to be faster and have smaller key sizes.
If it keeps its promises this will be huge. To be honest it's almost too good to be true (and to have such an open license). Looking forward very much to using this in future projects.
Excuse my ignorance -- I find myself fairly knowledgeable in crypto/security, but I'm confused by "Sodium also provides a secure, chroot()-resistant drop-in replacement for the arc4random() function family, including the ability to generate random numbers within a given interval with a nearly random distribution."
Take a look at arc4random() implementations for a hint.
The generator has to switch to a new key after (in the OpenBSD implementation, too lazy to check other implementations now) 1600000 bytes of output.
How do you switch to a new key?
OpenBSD implemented a sysctl to retrieve the output of the kernel prg. This is great, as it perfectly works in a chroot()ed environment.
Other operating systems don't necessarily provide the same facility. They will try accessing /dev/urandom, and if it doesn't exist, revert to something pretty bad, like read whatever is on the stack and use that as a key.
Sodium provides randombytes_random(), randombytes_uniform() and randombytes_buf() that you can use as drop-in replacements for arc4random(), arc4random_uniform() and arc4random_buf(). On Unix, the file descriptor to /dev/urandom will be kept open, and accessible after a chroot() call. On Windows, the crypto services provider will be transparently used instead.
Hmm linux provides AT_RANDOM from the kernel at program startup usually for randomization of stack canaries (but it can also be used for other things).
This is great, especially the Python bindings. I've been looking for something similar for a while. I was hoping to package just a reference (portable) implementation of 'crypto_secretbox', but the code seemed to be split up in multiple files and I couldn't understand their build system...
SSH is a good example of the kind of thing that can't use high-level crypto libraries, because the constructions required to implement SSH are part of the specification.
We're all waiting for Daniel Bernstein to replace SSH, but it hasn't happened yet.
Do we need a replacement for SSH?
The OpenSSH team is doing an amazing work. Besides making the tool as secure as possible, they are also improving the protocol. The recently added support for encrypt-then-MAC modes is a great step forward.
I have a lot of respect for the OpenSSH team but think that they are saddled with a protocol that was designed (a) before anyone had a good idea how to design resilient cryptographic transports and (b) without a clear understanding of how people would want remote control channels to work in 2013.
Is that why, or is it because nobody can find it? At first I thought you were talking about google's Native Client, and it's the Google results for 'nacl' or 'python nacl', etc. have both the crypto lib. and the browser vm interspersed.
I'm the author of RbNaCl, the Ruby binding to Sodium. RbNaCl wouldn't be possible without Sodium, because RbNaCl is written as a Ruby FFI binding and thus requires a shared library. Sodium builds a shared library. NaCl does not. Let me explain why...
There are many problems that hinder building language bindings to NaCl. See, for example, node-nacl:
"WARNING: This library DOES NOT WORK on 64-bit systems, and there's nothing I can do about it before the next version of NaCl is available."
Much of the assembly in NaCl is presently NOT position independent code which causes many of these sorts of problems. djb has admitted as much and sought to fix these sorts of problems.
Here is what djb had to say in email recently:
"More language support. The real work here is making everything PIC. Of course, if what matters is the API rather than speed, then achieving PIC is easy: just remove the asm."
At the very least, djb recognizes that removing the ASM is a simple and valid option for achieving PIC now without additional work on the assembly code. In the meantime he is doing a lot of that work inside SUPERCOP.
There's also the elephant in the room: Windows. NaCl doesn't support Windows. NaCl will likely never support Windows. Sodium supports Windows.
Part of the problem here is that DJB has more than one research goal, and the most important of those goals is making this stuff as fast as possible; his omnibus goal is to get high-security crypto implementations so fast that they can be deployed universally in every application.
As a result, all this stuff is excruciatingly tuned.
Yeah, ASM is also probably the only realistic way you can guarantee constant time (i.e. guarantee a C compiler won't do something crazy behind your back)
In the meantime, people continue to use OpenSSL because using NaCl is too hard
> WARNING: This library DOES NOT WORK on 64-bit systems, and there's nothing I can do about it before the next version of NaCl is available.
FWIW, that warning is...incorrect. Anyone who actually wants to use NaCl with Node.js (and we do) can trivially get a 64-bit build (it's even done through Python). I don't disagree that the original author of node-nacl wasn't skilled enough to do it though.
> There's also the elephant in the room: Windows. NaCl doesn't support Windows.
Say what? NaCl supports Windows just fine – it's just djb's 'do' build script that doesn't, along with how to get a random number. There's absolutely nothing to prevent anyone from using NaCl on Windows, if they are willing to integrate it into a normal Windows build system (which is...trivial) and make a few, trivial, API updates.
----
Having addressed those mis-understandings, Sodium is a welcome development and will certainly make it easier for people to develop bindings to NaCl.
Yeah, cryptography libraries in general are a pain to use and typically require a lot of seemingly arcane configuration and confusing (to the novice) setup.
I wrote this library recently (primarily to scratch an itch on another project), which really does nothing more than pass sane defaults to PyCrypto and eliminate crypto jargon:
Stop using this and start using Sodium, which is more secure than your wrapper library in significant ways that illustrate why people shouldn't think of this stuff as "jargon" that they can just write wrappers for; for instance:
* It generates encryption keys insecurely instead of using a cryptographically secure KDF
* It leaks timing information on the MAC comparison
* It makes verification of messages optional; verification should never be optional (in your case, when verification is disabled, I think you have the CBC padding oracle vulnerability)
Thanks for your comments and recommendation. Your points (and my own failings here) illustrate why libraries like Sodium (which was not on my radar a few weeks ago) are a good thing.
As an ordinary developer, I just want to use a library like PyCrypto in a safe manner. When I described the language as "jargon" I did not mean to be dismissive of it, but rather to say that libraries like PyCrypto force you to make decisions about terms that are nothing more than jargon to a non-expert -- when really what a non-expert needs is an extremely simple API which makes those decisions on your behalf in a safe manner.
Thanks again and these are humbling lessons to learn.
You seem smart and like someone who enjoys this stuff, so if you want a much more fun lesson (or, uh, 40+ of them), drop a line to sean AT matasano DOT com and he'll send you a bunch of exercises that'll teach you how to exploit this stuff.
Open invite! Send mail to sean, he'll get you started. We're not publishing them and we'll ask you not to circulate them, but instead to let anyone you know who wants to see them to just mail sean.
Alright, email sent. Would love to work through them with the rest of the team at work though, think it would be good for all of us to have a good understanding of how to break crypto systems.
Your key setup leaves a whole lot to be desired. (There's a very real reason that PBKDF2 exists.) Bad crypto libs like this are the EXACT reason that things like NaCl and KeyCzar were created.
Libraries like NaCL, Keyczar, and Cryptlib work by removing all the design choices from cryptography. You don't pick the key sizes, you don't pick the algorithms, you don't even pick what kind of keys you exchange. They implement a whole cryptosystem, as if for some new unreleased version of PGP, subtract the file format, and present it as an API. They're great. They are literally the only way you should be deploying cryptography in your applications.
I think this is a great development, but I am not qualified to say. This sits in a weird place in between a language binding for NaCL and a fork of NaCL. As a bindings package it's an overachiever; as a fork, it seems to have made extraordinarily conservative changes.
You'd really like a real crypto dev --- Matthew Green, Colin Percival, DJB (NaCL's primary author; @hashbreaker on Twitter), Steve Weis, Trevor Perrin, &c --- to say "this is all totally sane, and if you buy the premise of NaCL, go ahead and use this instead.
You'd also really like Sodium to say "this is as much as we're changing from NaCL and nothing else", because what makes NaCL worth building on is the expertise that went into designing it in the first place, which, like me, Frank Dennis probably doesn't have.