git rev-list HEAD | measure formally `Measure-Object -Line`

jamal-kumar · on Sept 1, 2022

I need to get better at powershell but it keeps on annoying me massively with basic stuff you can do in unix shell since the 80s, 90s or 2000s that you still can't do in PS. It needs to reach a sort of usability parity for sure.

One of those things I ran into recently was trying to find an equivalent of the following thing I do a lot of in *nix programming: Finding files of a certain pattern in grep, only list them, and then pipe that through something like awk or perl to find/replace in each file for each of those found files in that given grep. That's super easy in *nix. Powershell still doesn't have such an equivalent. It came of age in a time when everything as text and etc from unix principles weren't considered.

mgdlbp · on Sept 1, 2022

I think the philosophy is that as much as possible is done by composing cmdlets within the PS ecosystem, Select-String being kind of like grep and PS itself being the one language for all scripting. There's a lot that 'awk or perl' could map to; here's a (made-up) example of converting dates in certain files from 'August 1, 2022' to '2022-09-01':

  sls '(ERROR|WARNING)' *.log | %{$_.Path} | %{ (gc $_) -replace '\b\w+ \d\d?, \d{4}\b',{
      [datetime]$d = 0
      [datetime]::TryParseExact($_.Value, 'MMMM d, yyyy', [cultureinfo]::InvariantCulture, 0, [ref]$d)
        ? $d.ToString('yyyy-mm-dd')
        : $_
  } | Out-File $_ }

PS's advantage: leaning on .NET for doing things the right way.

The above has mostly top-level pipes like the Unix equivalent. You could also start with Get-ChildItem and skip the Select-String, instead having a conditional within a ForEach-Object, which certainly can masquerade as awk:

  ls -file|%{gc $_|% -begin{"Counting in $_";$n=0}{if($_-match'\b\d{4}-\d{2}-\d{2}\b'){$n++}}-end{"$n lines"}}

demonstrating that PS one-liners can be just as readable as those of Unix tools!

But, as you've identified, the pain in pipelines with native commands is real. No subshells or named pipes, naturally. All command output is parsed and re-serialized—anything binary, or with linefeeds, or UTF-8 without BOM is, as expected, silently corrupted. At least that's being worked on (https://github.com/PowerShell/PowerShell/issues/1908) -- the hindsight shell manages to have an immense amount of WTF locked in, turned into wontfixes by Microsoft's backwards compatibility. Tour the footgun arsenal: https://github.com/PowerShell/PowerShell/issues/6745

Oh, and the dreadful slowness. But is it worth it, to no longer fear self-pwn by file naming?

jamal-kumar · on Sept 1, 2022

I do like how it's a bit more semantic but find/replace for all instances of files that contain a search term in any .sh is like

  perl -p -i -e 's/term/replacement/g' | $(grep -rl term)

It's unbelievably fast and handy when you have multiple directories to recurse through or or tons of files. Anyone can remember that after using it like twice. Powershell requires like five lines of code just to get that done as you demonstrated. That's not feature parity at all. I also strongly feel it's way more readable than your example.

mgdlbp · on Sept 2, 2022

Oops, the PS ? and : actually need to be on the same line. Whiteboard fail. Likewise there shouldn't be a pipe before grep, right? So,

  perl -pi -e 's/term/replacement/g' $(grep -rl --include=\*.sh term)

Apples-to-apples, the date conversion with perl is,

  perl -pi -e 'use Time::Piece; s/\b(\w+ \d\d?, \d{4})\b/
      eval{Time::Piece->strptime($1, "%B %d, %Y")->strftime("%Y-%m-%d")} or $1/eg' \
    $(grep -ilE '(ERROR|WARNING)' *.sh)

and the simple replace with PS is

  ls *.sh -r -File | %{($_|gc) -replace 'term','replacement' | Out-File $_}

Not Perl terseness, but not 5 lines either (maybe in Visual Basic).