Remotely-related question: how can you scale out a service that uses ZFS, so as to have N instances of that service on different hosts, but let them all use the same ZFS pool?
Can this be done with using something like NFS+ZFS?
Assuming those instances all need shared access to that storage, you would have to build some sort of shared storage on top of the ZFS solution. This might mean NFS shares, or a relational database, etc, depending on the nature of the shared data. You still end up with a single point of failure at the storage layer, and a potential performance bottleneck.
If the data they need shared access to is immutable (for example, the static files served by a webserver) I would much sooner bake them into a versioned container image and share that. It'll perform better by virtue of being on unshared local storage and eliminate the single point of failure/bottleneck.
Another option for eliminating the single point of failure and performance bottleneck is a distributed filesystem like cephfs or glusterfs.
I haven't used ZFS, but you can easily use NFS to share a filesystem from a central (ZFS) "file server". NFS works very well over wireguard, as wireguard gives you "trusted" source IPs, and NFS uses that to check permissions (It would also work well over yggdrasil/cjdns).
I've been wondering the same for distributing storage arrays around multiple sites, with local caching. Ceph seems to be the way.
Scaling out a _service_ that uses data managed by ZFS is where NFS helps you. You simply mount the same ZFS storage in multiple hosts via NFS.
If you meant scaling ZFS itself such that the shared storage is resilient to a full host failure, I’m afraid ZFS is not your best bet. ZFS scales vertically very nicely. Heck, I have a single host with 2 X 100G NICs and 4 x 40G NICS fronting 1.2 PB of ZFS managed disks. If that host goes down, say from a motherboard failure, all 1.2 PB becomes inaccessible until I can hook up the JBODs to a different host physically.
A ZFS pool is something only one host at a time can use. You don't share the pool, except maybe for redundancy with enterprise disks that can attach to two hosts; I'm not sure if OpenZFS or OracleZFS have features for that, or if you'd need to do it yourself.
You can share a filesystem (see the article for one way to configure that) or a volume (like a disk image, could be used for diskless hosts; iSCSI is a standard way of doing that).
Multiple services can mount a network volume over NFS/CIFS/SMB. The pool is not something you’d see or deal with outside of the host that’s operating the pool. It’s a local concern. Externally you’d expose network shares.
Can this be done with using something like NFS+ZFS?