The "copy-on-write borrowing" on write() is where the problem lies. To turn the writable pages into COWs you have to update the process's page tables to remove write permission, which requires a TLB flush. These kinds of games are a net loss, at least on modern hardware.
(And then the app likely reuses the buffer for the next `read()` anyway, requiring either replacing the page or faulting in a fresh one and doing the C in COW).
Huh. Like to see some numbers on that. Not arguing, just a little surprised. So you have data that shows bcopy() to be faster than the copy-on-write stuff?
As for the buffers, known problem, all the apps that used this sort of thing cycled through N buffers for exactly that reason. I think N=4 was way more than enough.
For file I/O, if you used O_DIRECT, then the data was dma-ed to/from your pages directly, never went through the kernel's file cache at all.
All the file performance came from getting many disk DMA engines running in parallel.