On: Pipes and Filters

[brianpgordon]

I hate to be "that guy" but someone has to say something about the Useless Use of Cat.

http://www.catb.org/jargon/html/U/UUOC.html

[djur → brianpgordon]

I've never agreed with the UUOC concept when applied to pipelines. Using cat in a pipeline for a single file makes the flow clearly unidirectional, prevents certain types of errors like switching > and <, allows the command to be easily modified to handle multiple files or a glob, and frankly just seems easier to read.

Considering the tiny overhead of an additional cat process, UUOC these days feels like nitpicking.

[_pvxk → djur]

Also, the cat makes for faster testing. I tend to start out with e.g.

    head bigfile.1.txt | grep | awk | stuff

and refine things, and when output looks right, it's a simple "Ctrl+A Meta+D cat RET" to run it on the full output. Or vice versa if I suddenly want to go back to testing part of bigfile (or exchange the cat for "grep something").

If I want to change that to "< bigfile.1.txt", I have to "Ctrl+A Meta+D < Meta+F Meta+F Meta+F Ctrl+D Ctrl+D" – the extra keypresses are to delete the first "|" symbol. And if I suddenly want to change it back to head or grep, I have to reinsert the | (also I often by habit do Meta+D instead of Ctrl+D at the beginning of the line, which doesn't work as intended if the first token is "<" instead of "cat").

Those useless cats are quite handy when doing a lot of shell work.

[JetSpiegel → _pvxk]

You can use <bigfile.txt head | grep | awk

[_pvxk → JetSpiegel]

So if I want to change that to the whole file, I "just" have to Ctrl+A Meta+F Meta+F Meta+D Ctrl+D Ctrl+D RET. That's not really an improvement – especially since it depends on how many dots or similar are in the filename.

Also, that's a Useless Use of head, since grep has the option "-m10"

[JetSpiegel → _pvxk]

grep -m "Stop reading a file after NUM matching lines", while head -10 takes the first 10 lines and searches on them. Different things.

[_pvxk → JetSpiegel]

doh! you're right, I wasn't thinking :-)

[rdc12 → djur]

But then the article has this

"(If we move grep to run immediately after cat, and before putting data into Redis, this operation runs more than 1,200 times faster.)"

Which does indicate that in some cases the UUOC is justified (thou in this case the cat remains)

[davidu → brianpgordon]

I believe in this case, based on the author clearly having some mastery of the art, it was used to extend the example for the sake of explanation.

There are lots of ways to tighten up the example, if needed.