Would the Core Infrastructure Initiative be willing to sponsor a cryptographer to review it?
Getting this right would make it a lot easier to have faith in encrypted backups. Right now, you have to trust that each network backup product like bacula is doing it right.
I just randomly picked bacula and then looked up what they are doing. They're using OpenSSL with AES-CBC. I don't think anyone would recommend that.
As a start, it would be nice if Datto could document the design decisions. For instance, AES-CCM is a bit of a strange default and it's not explained anywhere. I see Solaris chose AES-128-CCM and maybe that influenced why OpenZFS is using AES-256-CCM. I haven't found a cryptographer in favor of CCM over GCM. For instance, Matthew Green doesn't appear to like it over GCM.
Given that GCM IV reuse is catastrophic and one of the main use cases for this is backups to potentially untrusted sources, I'm curious if AES-GCM-SIV would be a more conservative choice.
I'm Tom Caputi, author of the ZFS encryption patch and I can answer of few of your questions.
First of all, the choice of AES-CCM. I have had a few people ask me why we didn't chose something like ChaCha20 as a block cipher instead of AES. This is largely because AES is by far the most scrutinized block cipher around. It's use is currently so widely accepted that modern Intel CPUs have built-in AES instructions to improve performance. While its true to say that ChaCha20 (or other block ciphers) might theoretically be faster or that they ARE faster on some architectures like 32-bit cell phone CPUs, this is not currently the case with the vast majority of ZFS deployments.
As far as the choice for CCM as a default goes, this one was a little bit harder. Originally this decision was made to match the Oracle implementation as much as possible (a design decision which has since been dropped). Later, when we re-evaluated the descision, we found a paper (http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/comment...) indicating there might be weaknesses with the authentication mechanism, although the paper only mentioned cases with truncated authentication tags. So in the interest of being as conservative as possible, we chose the option which looked the most secure. We did not look into AES-GCM-SIV since it is very new (it looks like it actually came out this year) and so I would not by any means consider it a "conservative" choice.
As far as performance goes, we have not yet (as far as I'm aware anyway) seen a case where read or write speed wasn't bottlenecked by the disk speed. The benchmarks you posted are (as far as I can tell) single threaded and ZFS processes each block asynchronously.
The biggest thing here is that AES-256-CCM is only the default. It is easy for users to pick GCM for the time being and for developers to add newer, better encryption algorithms and change the default as time goes on. I wouldn't be surprised i we had changed the default by the time that the patch ends up in a tagged release.
I'm not a cryptographer, but for your dedup version, you might want to consider using SIV mode [1,2]. It provides a well known, already vetted deterministic authenticated encryption, so you don't have to build your own from HMAC etc and then get it vetted.
Thank you for the suggestion. As I said above, there is nothing preventing something like that from being implemented in ZFS in the future. Unfortunately, however, the encryption implementation uses a port of the Illumos Kernel Crypto Framework, which has not yet implemented an SIV mode.
As far as using our own HMAC goes we are using a standard SHA512-HMAC implementation which should be just as secure.
