On: For the Love of Pipes

[msla]

I find something like this:

   grep '^x' < input | sed 's/foo/bar/g' 

to be very readable, as the flow is still visually apparent based on punctuation.

[OskarS → msla]

I don't like this style at all. If you're following the pipeline, it starts in the middle with "input", goes to the left for the grep, then to the right (skipping over the middle part) to sed.

     cat input | grep '^x' | sed 's/foo/bar/g'

Is far more readable, in my opinion. In addition, it makes it trivial to change the input from a file to any kind of process.

I'm STRONGLY in favor of using "cat" for input. That "useless use of cat" article is pretty dumb, IMHO.

[dearrifling → OskarS]

Note that '<input grep | foo' is also valid.

[kps → msla]

In this particular example, ‘unnecessary use of cat’ is accompanied by ‘unnecessary use of grep’.

    cat input | grep '^x' | sed 's/foo/bar/g'

→

    sed '/^x/s/foo/bar/g' <input

[irishsultan → kps]

That's not the same thing. The sed output will still keep lines not starting with x (just not replacing foo with bar in those) where grep will filter those out.

[kps → irishsultan]

Yeah, Muphry's law at work. Corrected version:

   sed -n '/^x/s/foo/bar/gp' <input

This may be an inadvertent argument for the ‘connect simpler tools’ philosophy.

[sigjuice → msla]

You can just remove the <

[msla → sigjuice]

You can if input is a file. It might be a program with no arguments or something else.

[sigjuice → msla]

In your original command, how can 'input' be a program with no arguments?

[msla → sigjuice]

Oh, damn. You're exactly right.

OK, to save some of my face, this will work:

    grep 'foo' <(input) | sed 's/baz/bar/g'

... at least in zsh and probably bash.

[laumars → msla]

I don’t like that at all. That creates a subshell and is also less readable than

    input | grep foo | sed ...

[msla → laumars]

That specific example is less readable, but I do like being able to do this:

    diff <(prog1) <(prog2)

and get a sensible result.

And sometimes programs just refuse to read from stdin but do just fine with an unseekable file on the command line. True, you do have this:

    input | recalcitrant_program /dev/stdin

... but it's a bit of a tossup as to which one's more readable at this point. They're both relying on advanced shell functionality.

[laumars → msla]

> That specific example is less readable, but I do like being able to do this:

> diff <(prog1) <(prog2)

> and get a sensible result.

That is called process substitution and is exactly the kind of use case that it's designed for. So yes, process substitution does make sense there.

> input | recalcitrant_program /dev/stdin

> ... but it's a bit of a tossup as to which one's more readable at this point. They're both relying on advanced shell functionality.

There's no tossup at all. Process substitution is easily more readable than your second example because you're honouring the normal syntax of that particular command's parameters rather than kludging around it's lack of STDIN support.

Also I wouldn't say either example is using advanced shell functionalities either. Process substitution (your first example) is a pretty easy thing to learn and your second example is just using regular anonymous pipes (/dev/stdin isn't a shell function, it's a proper pseudo-device like /dev/random and /dev/null) thus the only thing the shell is doing is the same pipe described in this threads article (with UNIX / Linux then doing the clever stuff outside of the shell).