A mirrored ZFS setup with one offline backup and one cloud backup will prevent nonsense like this.[1] If you know what you are doing, you can do a lot of stuff with ZFS even on a single disk.[2]
The problem is I get home at night and I want to spend time with my kids, not admin my ZFS system. Which is why I'm running a really old version of FreeNAS, it has owncloud (not nextCloud) which fails to start since the last time I updated freeNAS, and I haven't figured out how to make it work again - mostly for lack of time/interest in doing it.
There is still administration required. Sure ZFS itself is easy, but the rest of the ecosystem on top of that is hard. You need a lot more software than zfs to make something useful.
I am not sure. Quite possible bad data was written over SMB, not necessarily that the drives failed. As you see in the article, the damage to images is systemic and drives don't seem to be failing. More like Lightroom/macOS/macbook RAM/Samba wrote (accepted) bad data.
His entire approach to storage of critical data was a disaster waiting to happen. Storing data on FAT-formatted USB drives and then transferring them to a Btrfs file system over Samba? Why?
If you run an OS with first-class ZFS support and your files arrive on the platform as soon as possible over as few intermediaries as possible, the chances of such mishaps are greatly reduced.
Okay, let’s see how we can practically improve this (I am interested in a good solution as well):
1. Camera stores images/videos on an SD card.
2. When the card is full, images are copied in the field on the USB drive. I imagine the user has a 128G SD card and multiple photo sessions will exhaust internal storage quickly, especially on a mac with 256G-512G SSD, hence an external drive. APFS may be used instead of FAT but then the drive won’t be readable on Windows or Linux. Mac doesn’t allow writes to an NTFS filesystem unless you buy a 3rd party driver of unknown reliability. I guess one can try to combine your linked guide with https://github.com/spl/zfs-on-mac to get ZFS on a USB drive but all of that is done at your own risk.
3. When the user gets home, they can plug the drive into a Linux/BSD machine with ZFS. They copy the files locally.
4. They proceed with their favorite workflow on another OS accessing files on ZFS over SMB. Bad writes from macOS over the network can still bork files on ZFS, though you can apply a readonly policy to RAW images.
Step 2 and 4 still look quite hard to work around for me. Bad SD card reader and bad RAM or bugs in any of the 3 machines are still a risk.
> get ZFS on a USB drive but all of that is done at your own risk.
I don't consider ZFS on portable devices to be more risky than FAT32. You can always copy the files to two devices instead of one. Or carry around multiple SD cards.
> though you can apply a readonly policy to RAW images.
Snapshots will protect you if files are corrupted after they make their way to the file system.
> Bad SD card reader and bad RAM or bugs in any of the 3 machines are still a risk.
Sure they are. But they are not in your control (except RAM, where you can always pay for ECC). I would worry about things that are.
[1] https://www.backblaze.com/blog/the-3-2-1-backup-strategy/
[2] Forbidden Arts of ZFS | Episode 2 | Using ZFS on a single drive [https://www.youtube.com/watch?v=-wlbvt9tM-Q]