I guess zfs send and similar are better solutions, but what if we could query the filesystem for existing checksums of a file and save IO that way, if filesystems on both sides already stored usable checksums?
> I guess zfs send and similar are better solutions.
It depends. I recently built a new zfs pool server and needed to transfer a few TB of data from the old pool to the new pool, but I built the new pool with a larger record size. If I’d used zfs send the files would have retained their existing record size. So rsync it was.
Unless you are also doing FS-level deduplication using the same checksums, it generally makes no sense for these to be cryptographic hashes, so they're not necessarily suitable for this purpose.
IIRC neither ZFS nor btrfs use cryptographic hashes for checksumming by default.
Strangely enough SHA-512 is actually (50%) faster than -256:
> ZFS actually uses a special version of SHA512 called SHA512t256, it uses a different initial value, and truncates the results to 256 bits (that is all the room there is in the block pointer). The advantage is only that it is faster on 64 bit CPUs.
It seems that SHA512t256 is another name for SHA-512/256. It's a shame that the initialization is different from SHA-512, as it would have been very useful to be able to convert a SHA-512 hash into a SHA-512/256 hash.
ZFS send does not do any kind of cross checking with the receiving end so no, not really ideal. Even incremental ZFS sends don't do this, it keeps state on the sending side only. It's ok for its intended usecases but it's not a direct rsync replacement.
It's also hard/impossible to restore individual files out of a ZFS send stream without restoring the whole thing so I've reverted to using tarballs of ZFS snapshots for backups instead of ZFS send. Again, it was never really meant for this so it was my mistake trying to use it that way.
