On: When Bloom filters don't bloom

[majke]

`sort -u` is slower for me.

Doing cat from left to right helps readability - it's important.

Performance-wise cat gives you 64KiB blocks of data, while direct pipe can give more. My programs (mmuniq-*) use 512KiB input buffer, so indeed with redirection you can reduce the number of read/write syscalls 8x, but it doesn't change much of the timing frankly.

Parallelization is an interesting aspect, which we didn't discuss really.

[adrianmonk → majke]

> Doing cat from left to right helps readability - it's important.

People may balk because it's unfamiliar, but this is syntactically legal:

    < logs.txt sort | uniq > /dev/null

That is, the redirection customarily goes at the end, but it doesn't have to.

EDIT: Also, in this specific case, the "sort" command can take a file argument, so you can also do this:

    sort logs.txt | uniq > /dev/null