_that_do not fit into memory_ is an important caveat to your abandoning the speedup offered by SLOG for sync writes.
Case A : No SLOG.
You have sync writes and they hit memory, the transaction groups get flushed to disks and _only then_ the ack goes back completing the write IO operation.
Case B : low latency SLOG.
You have sync writes and they hit memory, immediately gets written to SLOG, immediately the ack goes back completing the write IO operation while batching in transaction group does the flush to disks in background(at slow disk speeds). Sure, if you are out of memory to commit those write IOs, you're going to hit the disks in hot path and get undesirable perf.
Case B surely is always better than case A wouldn't you say?
I agree with your statement above completely. You understand ZIL SLOG perfectly. I have an NVMe as my ZIL SLOG device but it really doesn’t help much for my workloads - which involve transfer of large files around.
In my scenarios were I ran out of memory on large writes and them get slow downs I would love a real write cache on an NVMe or tiered storage. It would make a huge difference.
Case A : No SLOG. You have sync writes and they hit memory, the transaction groups get flushed to disks and _only then_ the ack goes back completing the write IO operation.
Case B : low latency SLOG. You have sync writes and they hit memory, immediately gets written to SLOG, immediately the ack goes back completing the write IO operation while batching in transaction group does the flush to disks in background(at slow disk speeds). Sure, if you are out of memory to commit those write IOs, you're going to hit the disks in hot path and get undesirable perf.
Case B surely is always better than case A wouldn't you say?