Kernel developer here - honestly, I'm not surprised that fsync() stops the world, and furthermore I suspect that even if you got past the vfs layer, you would see different effects on different filesystems (i.e. you still couldn't bet that fsync() would act like you want it to). The semantics of fsync mean "Please guarantee that everything is written to disk, flush all caches now".
The kernel doesn't keep a 2nd queue for post-fsync writes that it will then swap into the "real" one - think about what happens if someone else calls fsync(); does it spin up a 3rd queue for that one? Does the fsync block? I think it would quickly descend into Crazyville.
Yes I guess the implementation may get more complex, not sure about the actual implementation it's just a linked list of operations to flush like it happens looking at a few source code fragments, then it's just possible to put a "sentinel" in the list that will block the first fsync() when the first sentinel is found and so forth.
I mean, I'm all against the complexity myself in the code I write, so I can't really question this behavior and the "fsync every second" policy is not a huge use case indeed, but still it's important to now that. Googling a bit around there are tons of people that appear to be pretty confident that moving fsync() into another thread is the way to go, while instead stuff are working in a different way.
A pair of queues should be enough. The first queue is the one being fsynced, the second queue is open for writes. The first fsync call starts the first queue flushing. The second fsync call marks the second queue as read to flush once the first one is done. The third has no additional effect. Writes can still proceed against the second until it actually starts flushing. At which point the queues swap and you're back where you started.
The kernel doesn't keep a 2nd queue for post-fsync writes that it will then swap into the "real" one - think about what happens if someone else calls fsync(); does it spin up a 3rd queue for that one? Does the fsync block? I think it would quickly descend into Crazyville.