PowerShell is my guilty pleasure of the computing world. Once you've piped strongly-typed objects around between your shell commands, text scraping seems barbaric by comparison. The problem, of course, is that you can't use all that power for much, since the pool of PS-compatible tools and utilities is much shallower than it is for Unix shells. I'm really hoping this will help spur a new wave of PowerShell-compatible tools.
Come to the dark side, we have (strongly-typed) cookies.
After a number of years using PowerShell, my conclusion is the opposite: text "scraping" is just better for most cases.
Normal shell usage means doing a lot of one-shot pipelines incrementally. This is just easier and faster to do when passing text around because you can look at it and not have to inspect some object for it's type and what attributes/methods it provides. Parsing the text is not the problem here (although many people think it is), reasoning about what we're trying to do is.
And the over-verbose syntax doesn't help.
I compare this to human languages. It would be tempting to create a language with minimum ambiguity and clear cut concepts, but it wouldn't be practical. I guess PowerShell as an interactive shell is somewhat like this.
For automation, PowerShell is nice. The language sits between a shell script and going the Perl/Python route, but I still prefer shell scripts for simpler things and Perl/Python for more complex tools.
Having said this, PowerShell is the only sane choice on Windows and has made my life easier by no small amount. I never enjoyed managing Windows servers with their focus on GUI tools and their terrible CLI. PowerShell changed that and even "invaded" products like SQL Server and Exchange making them also nice to manage.
I'm not entirely convinced that "plain text" is "simple." For one, utf-8 is a variable-length encoding, which can cause all sorts of subtle bugs and sometimes leads to security issues because somebody failed to parse a character correctly somewhere. On top of this, using plain text means that every program has to choose its own control characters, essentially an encoding within an encoding. This is great for readability, but it's not always easy to know how a certain program will treat certain corner cases, or even really just at all. I'm not sure what you mean by "you can just look at the text," because unlike something that encapsulates functionality alongside the data, text input into a program might do anything.
For one, utf-8 is a variable-length encoding, which can cause all sorts of subtle bugs and sometimes leads to security issues because somebody failed to parse a character correctly somewhere.
UTF-8 is nice in that extended characters, despite consisting of multiple bytes, will never contain a low-ASCII character amongst them. Unless you're dealing with byte offsets, splitting and scanning UTF-8 strings by delimiters works just like ASCII.
because unlike something that encapsulates functionality alongside the data, text input into a program might do anything.
On the other hand, that "encapsulated functionality" is even more hidden and non-obvious. Unlike a stream of bytes that you can easily inspect simply by dumping them into a file, PowerShell objects are far more opaque entities.
Is there no trivial way to print a PowerShell object? As JSON or XML or an ascii table formatted into columns? That seems like a point of friction, then, I'd agree.
By default, an object at the end of a pipeline will be printed to the screen in tabular format.
(australis) ~\Desktop % Get-ItemProperty e
Directory: C:\Users\Dustin\Desktop
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 8/12/2016 9:06 AM e
I believe types can specify which columns to display by default; if you want more info, there's always `Format-List`:
(australis) ~\Desktop % Get-ItemProperty e | format-list
Directory: C:\Users\Dustin\Desktop
Name : e
CreationTime : 8/11/2016 10:58:56 PM
LastWriteTime : 8/12/2016 9:06:13 AM
LastAccessTime : 8/12/2016 9:06:13 AM
Mode : d-----
LinkType :
Target : {}
It's also possible to format any object via `ConvertTo-JSON`:
> I believe types can specify which columns to display by default;
Indeed, the default view (table/list/custom) can be specified, and for the view the default properties can be specified.
Consider how 'ls' produces a sequence of FileSystemInfo objects (when run against a disk file system). The view specifies that the list be grouped by "parent".
However, the underlying stream is still just a continous stream of FileSystemInfo's. It is the terminal display format definition that causes it formatting to break and write a directory heading whenever the next item has a different parent from the previous one.
You can certainly print them, but there's no guarantee you'll get all its contents by default, and there's still the problem of how to create those objects from some other output, e.g. in a file.
Or it can mean a lot harder to understand and use. With types you get all of the metadata that you might not even have if it was plain text. In addition, every application chooses it's own serialization format. In the end, the only programs that compose well are those that work on text, not structured data (i.e. grep, awk, sed, etc...)
> You can certainly print them, but there's no guarantee you'll get all its contents by default, and there's still the problem of how to create those objects from some other output, e.g. in a file.
That's what CliXML is for. Export-CliXml writes objects to a file. Import-CliXml reads them back.
CliXml (Command Line XML, IIRC) is a serialization format for PowerShell objects. It is the format used to transfer objects to/from remote machines when PS remoting is used.
Granted, the re-hydrated objects are not the original objects. Most method will not work after re-hydration as the object has been ripped from it's context. Think ProcessInfo (the objects returned from 'ps' (Get-Process)) - in it's original form it can be used to interact with the process, e.g. terminating it. In the rehydrated form it can be used only to inspect properties.
Speaking from experience, this is rarely a problem. CliXML works remarkably well.
> Normal shell usage means doing a lot of one-shot pipelines incrementally.
Sure because no-one ever takes those "one-shot" pipelines full of `cut`s and `grep`s and distributes them in real products. /s
As far as I can tell, Powershell is designed for scripts that are meant to be distributed, so they must be robust. Normal 'text scraping' Bash is shit at that.
If we are not talking about interactive use then to distribute "robust" scripts, you could use nicer and richer Perl, Python eco-systems.
For interactive use, the conciseness, the human-readability of input/output and the fault-tolerance to unrelated changes in the input/output formats provided by Unix shells are also preferable.
The motivation for PowerShell was to have something good at both. Why switch contexts when moving from interactive use to coded automation if you don't have to?
Because the requirements change depending on which one you're doing: quick shell automation isn't the same as the kind of automation you'd use perl/python for.
And yet most of the time I see bash being used for the same kind of automation that you think is good for perl/python.
And not because bash is good, but because it is what people know (since they use it interactively) and because it is installed everywhere.
So if someone had a tool that was installed everywhere, and used interactively, that could also be used to create more robust automation tasks, that seems like a win to me.
It depends on how complex the automation is. I'd no doubt use a pipeline in places where you'd use perl/python.
As for interactive use and robust automation, Bash isn't as bad as you'd think. The reason I'd go to python is because of script complexity, not lack of robustness.
The KISS principle ("Keep it simple, Stupid") is the best way in many cases. In Unix you can quickly enter a pipe without special coding which does also non-trivial stuff. For instance,
gets a sorted list of all entries in all Excel files which contain the name "Miller", no matter how deeply the files are located in the directories. Can you do this in Powershell quickly? I don't know, I am actually curious.
Objects in pipes are convenient and powerful. However, for most applications of pipes they are probably overkill. If things get tough you can simply use files rather than objects.
I would not be surprised if many Windows users will prefer bash pipes from the Ubuntu subsystem in Windows 10 rather than Powershell because it is much more handy for most pipe applications.
There's another option however, where you can leverage Excel itself (granted, this is likely to be a Windows only approach):
$excel = New-Object -com excel.application
And you can now open XLS/XLSX files and operate on them as an actual excel document (including iterating through workbooks/ worksheets, etc.). It's all just objects.
Thanks for pointing this out. However, your example is not equivalent since "ls" searches only in the current directory. Nevertheless it is good to know that it's basically possible since it helps a lot to manage mixed Windows/Linux networks.
> I compare this to human languages. It would be tempting to create a language with minimum ambiguity and clear cut concepts, but it wouldn't be practical.
Lojban[1] is usable in practice (though that may be because it still allows you to be ambiguous when desired).
a. You can still use the native binary just as well. Especially in this case.
b. There was no PowerShell cmdlet for it at the time. There also isn't one for executing Python code, you still have to call python. Or, if you're so inclined, load up IronPython's assembly, and use an overly verbose .NET method call ...
c. There now is a cmdlet for doing so: Expand-ZipFile.
I'm actually curious as to those complaints: Do you never write a function in bash to abstract away something? A function in other languages? Is everything just a series of crystal-clear one-liners?
Shell scripts in Unix-likes tend to glue together a bunch of other programs and (usually) not do much programming in the shell's language. You can write PowerShell just the same, actually. It is a shell, after all.
If you're used to cmd or bash, the jump in verbosity is very noticeable and definitely gets in the way.
It might be a cultural thing, because I'm no fan of C# and its typical style either; no surprise then, that I find PS syntax exceedingly verbose too. It just feels excessively bureaucratic and awkward to have to write so much. On the other hand, bash, awk, sed, and all the typical Unix commands and its associated ecosystem seem like they "get out of your way" far more effectively.
c. There now is a cmdlet for doing so: Expand-ZipFile.
That example already shows the verbosity increase clearly - why is it "Expand-ZipFile", and not "ZipFile-Expand", "Expand-Zip-Format-Archive", or something else? In contrast, "unzip" is short and easy to remember. The fact that a native binary might exist for a given task is irrelevant to the observation that the shell's language is itself more verbose.
I'm actually curious as to those complaints: Do you never write a function in bash to abstract away something? A function in other languages? Is everything just a series of crystal-clear one-liners?
Abstraction helps reduce code duplication but is not useful when each line of the script is quite different, and in that case PS remains more verbose. Ultimately, the overhead is still higher.
> Why is it "Expand-ZipFile", and not "ZipFile-Expand", "Expand-Zip-Format-Archive", or something else? In contrast, "unzip" is short and easy to remember.
Iff you know it already. PowerShell is built around a few conventions. One is that all commands share the Verb-Noun pattern. The verb always comes first. Then there are a bunch of common verbs that all share the same basic meaning, regardless of context. Expand is such a verb that is the opposite of the Compress verb. You may not like then choice of verb but there are always going to be names you didn't choose, so that's probably a petty point. In the end, PowerShell makes it easy to discover commands you may vaguely know exist. I'd argue that also helps remembering them once you know them. Just from knowing unzip you wouldn't be able to guess the command to extract a GZIP file, for example.
___________
P.S.: I have to retract part of my original post here. Expand-ZipFile does not exist natively in PowerShell. I stumbled across http://ss64.com/ps/zip.html and while that site does have documentation for all built-in cmdlets, I didn't read too closely and this was actually documentation for a wrapper around the .NET framework's own functionality. This does not change the discussion about the name and discoverability, though, except that New-ZipFile should probably use a different verb. Actually, the PowerShell cmdlets for handling ZIP files are Compress-Archive and Expand-Archive, exhibiting the properly mirrored verbs.
That was picking an example from my previous try to work with ps. I was not the admin of the server so no installation of extra programs. Or upgrading ps to the newest version.
The shell scripting I have needed has been typically very simple. Like schedule to copy this file from that server and run this command.(what was precisely all that I was trying to achieve the last time, only the file was compressed. Would have done with .bat, but that was even more difficult) As you say, using it as a glue and build the complexity to the programs behind the shell.
This is all a big part of the reason why I personally advocate using a standardized text-based object serialization format (like YAML) for these sorts of things. In particular:
* Still text based and human-readable, which helps for debugging/troubleshooting
* Still text-based and machine-readable, so it's inherently cross-platform (assuming that all said platforms agree on their text encoding)
* Still less troublesome than piping arbitrary text through `grep` or `sed` or `awk` or what have you
* Still provides the "everything is an object" benefit that's lost with arbitrary text streams
* Still orientable around streams of data, at least for YAML (by using the document delimiter)
Passing CSV around can be a useful middle ground, if you're using utilities that all do proper escaping and allow newlines in strings etc., such as csvkit.
The problem is that not all tools agree on this one, but: lines are separated by newlines, items are separated by commas. An item can be quoted (and this must be done if it contains a newline, comma, or double quote) by surrounding them with double quotes. Inside such quoted items, double quotes can be escaped by doubling them.
Yes, of course; that is one end of the "middle ground" I described. The problem is that in Unix, there is no common convention for how to escape the delimiter, so you cannot pass arbitrary strings. CSV can do this, with the proper tooling. Being able to pass around arbitrary strings is quite valuable, and although it is not quite the same as passing objects around, it has the advantage of still being human-readable text. That is why I called it a "useful middle ground" between Powershell and the delimiter-separated fields of Unix tools. However, you do need special tools such as csvkit, as Unix tools like awk, sort and cut cannot properly parse it.
Oh, you can pass arbitrary strings, you just can't pass arbitrary strings as part of compound datastructures. The closest you can get is using null or some other weird character to separate lines ($RS='\0' in awk), and something similar for field separation (like -0 in find, which is $FS='\0' in awk). But \0 is only one special character, so you need another one.
As a note, there have been strongly-typed shell languages for Unix for some time. Scheme Shell[0], Turtle[1] and TCSH[2] leap to mind; but there's nothing preventing a developer from using Perl, Python, Lisp or any other language that allows itself to accept stdin as an interpreted script.
This is why I find PowerShell to be an answer to a question nobody asked. If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl to do the heavy lifting.
PowerShell will never accumulate a library as comprehensive as what any of those three have, each has had decades to accumulate packages of all kinds, and more are still being added.
It's odd, but not surprising given their history of "Not Invented Here", that Microsoft never even considered using Python or Ruby when they both have .Net implementations. Their enthusiasm for JavaScript wasn't sufficient for them to embrace that either.
As far as I recall, the original impetus behind what became PowerShell (as handed down to us in a conference room by Ballmer himself, sometime around 2001) was to fill the gap between ops people who did enterprise administration manually with tools like MMC and engineers who automated enterprise administration with tools like C++/DCOM. The latter were necessary in a lot of cases, but they were expensive, and we needed to give the industry a way to do powerful automation without hiring a bunch of PhDs. So, yes, someone did ask for it - the IT industry.
I was part of the Hotmail team which consisted mainly of Solaris admins trying to mass administer a giant number of Windows servers. We kept pleading with the Windows team and upper management (forgot who it was at the time, Raikes?) to give us a powerful shell and ssh on Windows. We ended up licensing FSecure's ssh daemon for windows. We also used cygwin, too.
> So, yes, someone did ask for it - the IT industry.
... who couldn't use the well-known mature shell tools since they were locked into a closed platform, and Windows' cmd.exe was comparatively very weak.
It wasn't cmd that was the problem (although, it is weak); it was how Windows exposed APIs to allow automated solutions.
You had two options: VBScript (which is a glorified COM+ agent) or a "real" programming language like VB6/C# which mixed COM+ with Win32 API calls (which are too hard to use for most non-programmers).
Powershell within itself does improve CMD. But that isn't primarily how PS pushed Windows automation forward, Microsoft said that every major server feature and server function should work in Powershell, so engineers at Microsoft had to look at it (be it their lacking COM+ interface, provide a new PS interface, or something else).
The net result isn't that CMD got replaced by something better. It is that we have a lot more APIs and ways of interacting with Windows features and services than we ever used to.
The ironic thing is that today you could write MS Dos-style console applications that work in CMD which provide all of this new automation functionality because of the Powershell push. Back before Powershell you'd have to hack it using unsupported APIs, registry entries and similar.
If all this API effort had gone into a real cross-platform language (Python, Ruby, or even Perl which was king in 2001), Windows would have flourished. They went for their traditional lock-in approach instead, so now they have to catch up.
Late reply, sorry. I don't think we're on the same page: Windows sold relatively ok, basically doing tying the battle for the Enterprise sector, but in the meantime it lost the cloud war and was literally wiped out of several other segments (mobile, web etc). But don't take it from me: if everything were hunky-dory, they wouldn't be busy steering like crazy towards opensource, something that was unthinkable 15 years ago.
> are you suggesting that Perl would have been a better solution for a Windows interactive shell and automation tool?
From a mindshare/commercial perspective, yes it would have been (and I say this as a complete Perl-hater). By starting from scratch, MS then had to spend a decade evangelising their new solution, with mixed results. It would have been tremendously easier to just offer first-rate support for established sysadmin tools (ssh, perl/python etc) from day 1; but again, don't take it from me: the big new Windows Server features are Ubuntu-bundling and native ssh support. This would not be necessary, were Powershell and Windows dominant from a commercial perspective.
It's moving the goal posts a bit to suddenly restrict this discussion to only servers, no?
There are plenty of Windows servers running plenty of businesses. They may not outnumber non-Windows servers, but I don't think "majority" has to be a prerequisite for "flourish".
There are also buildings full of Windows clients...
This discussion is about PowerShell, which is primarily intended to automate server management and has been ported to Linux, presumably, for the same purpose.
> This discussion is about PowerShell, which is primarily intended to automate server management
Many (me included) find PowerShell extremely useful for automating any sort of Windows management, client or server. So I disagree that the discussion should be implicitly assumed to be limited to Windows servers.
I was administering windows computers at a university at the time of windows 98and most of my automation came from ActiveX where I could register external libraries and script most OS interfaces with vb or js. Powershell is just that plus lots of objects and types
No. The well-known mature shell tools manage files and processes - neither of which are a focus of managing a Windows system. Windows revolves around services which you RPC to, which internally store their state - attempting to faff with their state from outside the service is almost always both undocumented and likely to cause failure.
Powershell allowed those services to standardise their RPCs in such a way that sysadmins could call them easily. Alternatively, you'd have COM or a variety of other RPC mechanisms, depending on the service, often undocumented themselves as they were only ever intended to be used by a GUI packaged with the service.
It's been years since I used Windows APIs -- back then there was OLE Automation, which allowed dynamically, late-bound (call by name, really) RPC calls into COM from scripting languages. Has this been superceded by something else? Using IDL-defined COM interfaces from scripts sounds like something nobody would want to do.
Indeed, nobody would want to do that, which is why Powershell exists. Scripting VB or Javascript against COM or whatever else the software vendor decided to use was generally an awful experience, if the interface was even documented at all. Powershell replaces all of that for system administration purposes - you can even run Powershell commands easily from within any .Net application and get structured, object-oriented output, which is what at least all the Windows Server GUI stuff does now.
You're still stuck with OLE/COM for e.g. Excel automation, though, I think.
Well, lots of windows admins used things like cygwin and the like to get a unix-y working environment, but PowerShell offers a much more robust set of mechanisms to manipulate the underlying Windows operating system.
> who couldn't use the well-known mature shell tools since they were locked into a closed platform
Yeah, I remember the bad old days, when I'd download that CygWin installer and Windows would just give me the old skull and crossbones and say "You're not allowed to install GNU tools, Sucka!" One time I jailbroke XP and got it installed, but then CMD was all "You can't run grep.exe and sed.exe under CMD, Sucka! Not allowed!" I was so sad that I couldn't use those tools on Windows, so sad.
> a way to do powerful automation without hiring a bunch of PhDs.
I remember writing Windows software with DCOM was difficult, but it was more on the painfulness side than a difficult intellectual endeavor. Sure you didn't need PhDs to automate things.
> So, yes, someone did ask for it - the IT industry.
The part of it stuck in Windows, that is. Vendor lock-in is a powerful thing.
Yeah, DCOM was a task of fillout the correct fields with the correct values, and try again until you get it right. Anyone with passing C++ knowledge could do it through shear brute force.
It's one of the things which made me never want to program on Windows again.
I ported a bunch of COM to Linux in the late 1990's. Not DCOM with the full RPC, but just in-process COM. I had .so shared libs with DllMain and DllCanUnload now in them, I had CoCreateClass, I had a significant part of the registry API implemented in terms of a text file store (HKEY_LOCAL_USER being in the home directory, HKEY_LOCAL_MACHINE somewhere in /etc, ...) and such. IUnknown, QueryInterface, AddRef, Release, ...
Netscape did that with XPCOM [1] they once thought that this is a good idea - i guess for versioning of interfaces; nowadays firefox has moved on from this.
Haha, funny enough our company did this as well - we had lots of legacy C++ code using COM and wanted to port to Linux. It was not that hard after all.
That's actually a pretty substantial chunk of "the IT Industry" though. And if you're the people who make Windows, it's not surprising that it's a big enough chunk to do something about.
I personally find that the term "IT" has become a bit of a code word for "Enterprise Wintel", and that anyone working outside of the gravitational pull of Microsoft prefers to describe themselves as working in "tech"
That being said I really don't mind that MS went the PowerShell route. It improved their ecosystem a lot and I belive was also one of the engines behind the more open minded move towards open source...maybe someone at MS can chime in but I feel like PS was always one of the strong projects pushing that forward from talking to some MS engineers.
PowerShell, or at least the concept behind PowerShel, is an improvement/superset over the Unix shells -- and can do whatever the shells can do, plus stuff they cannot do, because they don't support typed entities.
Its problems are not technical or conceptual -- they are ecosystem and historical: lack of compatible tools, windows-only, etc.
And everybody remembers how Ford was forced to reintroduce a carriage with a horse a couple of decades later because nobody was really using those horseless carriages. Right?
I feel very similarly, but then I'm of the mind that if I can't apt-get it, I'm not interested, and I'll bet that "open source" doesn't mean to MS what it means to the open source world.
As for the NIH syndrome, one of my favorite Paul Graham essays is still "What Languages Fix" (http://www.paulgraham.com/fix.html) particularly for C#: the problem that C# was invented to solve is that Microsoft doesn't control Java.
C# incidentally solved many other shortcomings with Java. The languages are getting closer to parity, but for many years C# was pretty far ahead (if you only consider language features and not tooling/ecosystem). Between reified generics, better native interop, and many functional features, C# felt and feels more concise than idiomatic Java. And I've written plenty of both.
Disclaimer: MSFT employee expressing personal views
Java seemed to have a lot more options for deployments and monitoring for a long time. More advanced 3rd party libraries in general. It had simply been around longer and thus had those tools available. Now that C# has been standard for a while almost all of those tools have a C# equivalent.
As a .NET dev I'd actually argue that the things available to C# now are actually better (especially Visual Studio), but that's certainly biased.
> ...things available to C# now are actually better (especially Visual Studio), but that's certainly biased.
You call Visual Studio good? Have you actually used it?
VS2015 is painfully slow, and yes, I've installed all of the updates and I have no plugins or such. Heck, even Eclipse feels zippy compared to it. Something as simple as finding next text search match takes about a second. Changing to a different source code tab is also sluggish.
Context sensitive search (right click identifier -> Find All References) is useless for anything that is even a bit generic. Instead of actually analyzing context, it seems to simply grep all reachable files for a text string. Including SDK headers...
I use VS2015 only for C++, maybe that can have something to do with it. Maybe it works better for C#?
I also dislike how I have to login to use an IDE. Once it cost me 30 minutes of working time, I couldn't login to a god damn IDE that had an expired license. That's a great way to make developers hate your product and ecosystem.
It is a lot better for C#. All the features work perfectly for .NET languages, but because of the nature of C++ it's a lot harder to get these kinds of things working well with it. Some answers about why are here: https://www.quora.com/Why-is-C++-intellisense-so-bad-in-Visu...
Eclipse can run better on a potato but it's also very simplistic. You should also be able to turn off most of the features you don't want in VS.
Well, heck even Emacs with some LLVM wrappers can do better job at C++ Intellisense than Visual Studio. Eclipse's CDT runs circles around it as well (so much for being very simplistic). And don't get me started on trying to use Intellisense on large codebases - such as Unreal Engine's source code. It is just completely unusable (30+ seconds before the completion pops up, often blocking your IDE (a total usability no-no) with a modal "Intellisense operation in progress ...").
Also anything using complex templates (= pretty much any C++ code these days) throws it for a loop - making it useless where it would be needed the most.
And don't even get me started at the ridiculous state of refactoring support for C++ - even the stupid identifier rename doesn't work reliably, and that is the only C++ refactoring feature VS has (and we had to wait until 2015 for it!).
Sorry, but that something is hard doesn't mean it can't be done - especially when there are free tools which can do it and orders of magnitude better. Even Visual Studio can do it - if you buy the Visual Assist plugin.
Even the C# tooling isn't that great - yes, it does work better (I often do combined C++/C# projects) than the C++ one, but that isn't a particularly high bar to clear. Ever tried the Java tooling in Eclipse? Heck, that editor and the various assist features almost write your code for you ...
Where Visual Studio is great is the additional tooling - integrated debugger and the various profiling functions. Eclipse can't really compete with that, even though most of the tools (gdb, gprof, etc.) exist and have a lot of features that even VS's debugger doesn't have (e.g. reverse debugging, record/replay, etc.). Sadly, the Eclipse front-end doesn't support all of them.
However, the Visual Studio's editor and the tooling associated with actually writing and managing code (i.e. the part a programmer spends most time in) is stuck straight in the late 90's - just look at how code navigation (jumping to functions, navigating between files, projects, etc.) works in the ancient Visual Studio 6 and 2015 - it is almost identical.
There is also zero integration for alternative build systems like CMake. That is important for writing cross-platform code or even just basic sanity on large projects - the Microsoft's build configuration/project/solution system with millions of settings buried in cluttered dialog boxes is just horrid and error prone, especially with their myriads of mutually incompatible options - set one option incorrectly on one library and your project starts to mysteriously crash. Good luck finding that - and then spending hours recompiling your project :( However, nothing else is really supported properly. (Yeah, I know that CMake generates VS solutions/projects, but there is no integration within VS for it - so you have to be careful how you add files, how you change build options, etc.)
The saddest thing is that many of these issues are pure usability problems where changing/correcting a stupid design decision made a long time ago would reduce the programmer's frustration by maybe 80% already - like the modal blocking Intellisense operation windows, the ergonomic disaster that are the build configuration dialogs, the decision to show compilation error with templates on the deepest template in the expansion (i.e. somewhere in the STL header) instead of where it has occurred in your code, forcing you to mentally parse several pages of error messages to find the line number or just resorting the Intellisense matches to show you the project-relevant ones first instead of the unrelated stuff from the frameworks/header files so that you don't have to dig through it. I.e. common sense stuff.
C++ isn't going anywhere on Windows, no matter what MS and C# advocates say and it is really sad that the official tooling is in such poor shape.
I could continue ranting about many other things that are regularly driving me up the wall at work (like the incredibly slow and memory hungry compiler, horrid and unreadable error messages, especially for templates, etc.), but that would be pointless. IDEs are a matter of strong preferences and opinions, but I do believe that with Visual Studio, especially their C++ tooling, Microsoft has missed the boat many moons ago. And isn't even trying to catch up as it seems - they rather focus on adding stuff like Node.js support (are any Node programmers using VS or even Windows??).
However it does piss me off when I see people uncritically praising VS as the best thing after sliced bread and flat out dismiss the alternatives without even looking at them - when VS is really terrible IDE.
Unfortunately, this is something that is difficult to comprehend for someone who has never seen how it actually could work properly - either because they have never been exposed to the alternatives or because they have the VS-centric "workflow" (more like kludges working around VS silliness) ingrained so deeply that anything else will perturb them.
"Something as simple as finding next text search match takes about a second."
I'm not sure how our configurations differ but on a fairly modern machine (say, surface pro 4 i5) I've not had visual studio 2015 slow down, and editing and searching is az zippy as with Vim for me.
I've had the experience that some third party tools have bogged down the experience, though.
I run it at work on Xeon with SSD and 16 GB. Project sizes might be different. I don't have experience how the performance would scale with a project with tens of millions of lines of code, for example. (seems to be fine for me below that).
It does exhibit strange behaviour (including becoming suddenly very tardy) that only goes away by deleting the local cache files (intellisense db and whatnot) so I'm not claiming it's technically perfect.
It is meaningless to make such comparisons unless you also compare codebase sizes. VS is perfectly zippy if you have a small project - but try to hit CTRL+SPACE to complete an Unreal Engine or OpenCV identifier! You will be sitting there for 30 seconds or longer while VS is busy parsing/searching files ...
There's a box you can tick that says no to all of that stuff, though I don't know if you can access it after the first run. (I never have to log in when I run Visual Studio.)
Meh, to make Visual Studio do IDE-like things, people install things like ReSharper, built by the same people that are building IntelliJ IDEA.
And I worked with both and IntelliJ is much smarter, and has support for way more technologies. And yes, it's built in Java and can be sluggish, but then again I'm using it on both Linux and OS X and runs just fine on Windows too, so it doesn't tie me to an OS I don't want.
Also, I keep mentioning this, but .NET lacks a build tool like Maven or something similar (SBT, Gradle, Leiningen). .NET is still at the level of Ant / Make and .NET devs don't realize what a huge difference that is.
This used to be true but VS2015 is actually completely fine without ReSharper. Roslyn is amazing, I've even built my own custom code analyzers (ie. compiler extensions) that integrate in to the entire ecosystem seamlessly (nuget package registers them to VS project, IDE shows you code analysis on the fly, C# compiler uses them on every build, including CI/build server - and you can enforce stuff with it - eg. raise build errors).
>but .NET lacks a build tool like Maven or something similar
There are .NET build tools, FAKE, CAKE, etc. etc. but people don't use them much, tooling integration is notably missing. It would be nice to have something like Gradle in .NET but VS support for msbuild is good enough for most.
FAKE and CAKE are not replacements for Maven, but for Ant/Make. Maven and Maven-like tools on the other hand are so much more.
Here's a personal anecdote. I work on Monix.io, which is a cross-compiled Scala library for the JVM and Javascript/Scala.js.
I could easily do that because of SBT, which is Scala's flavor of Maven. Basically Scala.js comes with SBT and Maven plugins. And you can easily configure your project, indicating which sources should be shared between the JVM and Scala.js and which sources are specific. And afterwards SBT takes care of compilation for multiple targets, building JAR files for both the JVM and Scala.js. And then you can also sign those packages and deploy them on Sonatype / Maven Central. The tests are also cross-compiled too and running whenever I do "sbt test", no tricks required or effort needed to do it. And this project has a lot of cross-compiled tests.
And if I were to work on a mixed JVM/Scala.js project, I could also include a whole assets pipeline in it, with the help of sbt-web, another SBT plugin. SBT and Maven have a lot of plugins available.
The Scala.js ecosystem relies on Maven / SBT for build and dependency management. There's no "npm install" for us, no Gulp vs Grunt cluster fucks, no artificial separation between "real Scala" and Scala.js, only what's required. And trust me, it's a much, much saner environment.
Compare with Fable. Compare with FunScript. Don't get me wrong, I think these projects are also brilliant. But it's because of the ecosystem that they'll never be popular, because the .NET ecosystem inhibits reusable F# cross-compiled libraries.
I got into their stuff via pycharm and phpstorm and then ended up using intellij with the php and python plugins (plus node and a bunch of others).
I haven't enjoyed using a development tool so much since back when Borland was good, intellij is pretty much 99% of my development tool usage at this point and that 1% is mostly nano for quick 'I need to edit this one line' edits.
The thing they seem to get so right is that intellij with say php plugin feels like phpstorm and with python plugin like pycharm at the same time, it's not the features so much as how they are all seamless integrated and the tool as a whole feels designed.
One of the few bills I genuinely don't mind paying each month, everything thing else I've tried just doesn't compare*
No, FAKE is not a replacement for Maven. As I said, .NET is left behind in the Ant/Make era.
And since you mentioned an F# project, the absence of a Maven-like build, dependency management and deployment tool is why F#'s Javascript compilers are behind, because you have no shared infrastructure, no cross-compiled libraries, etc, because it's just too painful to do it.
>Also, I keep mentioning this, but .NET lacks a build tool like Maven or something similar (SBT, Gradle, Leiningen).
You mean in terms of integration with Visual Studio? Because both Maven and Gradle can compile .NET code (msbuild or xbuild). Gradle doesn't natively support Nuget's format, but there are plugins for it.
> .NET is still at the level of Ant / Make and .NET devs don't realize what a huge difference that is.
You can reference projects or other files that are in submodules. What about the .NET build process makes vendoring the only option?
Even so, it's always been more convenient in my experience when something just ships with fixed versions of dependencies already inside it. Those versions are known to work with the codebase and there's no chance of new bugs or other incompatible behavior being introduced from updates to the dependencies.
> AFAIK, even NuGet doesn't allow you to just check in the dependency spec, rather than the contents of all of your dependencies.
I have a project where I only checked in the solution's repositories.config (packages/repositories.config) and the project's packages.config (ProjectName/packages.config), and it seems to work fine.
I'd have to agree, though I spend most of my time in the node space, today via VS Code (though I don't use the debugger at all). VS Proper with C# has felt better to work with than any Java tool/project I've ever used, though haven't done much with that ever. I jumped on the C# bandwagon in late 2001, early 2002 when it was still in development. Started moving over towards node around 5-6 years ago, and am currently looking for a reason to work with rust or go.
Most of my issues with C# have always, like Java, been about "Enterprise" frameworks, patters and practices implemented where they aren't needed, and complexity for complexity's sake. VS + C# is pretty damned nice, if you can work on a green project and not bring in everything under the sun.
VS was probably always a better coding and debugging environment. I've used VS as far back as VS 2010 and it's always been great for C#, but I've never had more than a mediocre experience writing Java in an IDE. Did anything support VS's level of convenient debugging ability for Java?
I have not used VS as much as I have used Java IDEs, but I'd say they are on par. VS shines in how well the debugger is integrated with the rest of the environment, but both NetBeans and IDEA offer very close to the same level of convenience. Admittedly, I have never liked the Eclipse debugger.
> As for the NIH syndrome, one of my favorite Paul Graham essays is still "What Languages Fix" (http://www.paulgraham.com/fix.html) particularly for C#: the problem that C# was invented to solve is that Microsoft doesn't control Java.
That's not what Graham said. He said, "C#: Java is controlled by Sun", which is quite a bit different from Microsoft not controlling Java.
Microsoft is fine with using and promoting languages they do not control. C++, for example.
The problem with Java from Microsoft's point of view was that Sun did not want people to use Java as a mere programming language. They saw Java as a platform, and wanted people to develop for that platform instead of developing for Windows, or Mac, or Linux, or whatever. Sun wanted all access from Java programs to the underlying facilities of the OS upon which a particular instance of the Java platform ran to go through the Java platform.
Microsoft wanted something with the good parts of the Java language without the baggage and restrictions of the Java platform, and so they made C#.
》I'll bet that "open source" doesn't mean to MS what it means to the open source world.
That doesn't make sense. ASP.Net, .Net Core, F# are all good examples of open source projects. This announcement promises the same for Powershell: Development with the community.
What are you missing? What's the criticism?
If you want to see a broken open source project, check out Android instead..
except that C# as a language is by leaps and bounds better than Java (both in syntax and useful features departments), so there have been other problems to solve, too.
Its ecosystem isn’t. C# will never get accepted as an open source product. It only has open source code, it’s not actually ‘open-source.’ Compare it with Typescript, which—despite originating from Microsoft—is a truly open project, and getting love left and right.
C# might be better than Java/JVM, but it’s not better enough. The culture/ecosystem barrier is so high that C# would have to be miles ahead, technically superior in every way, to overcome it. I do hate the “worse is better” adage, but there’s no mistaking it, it applies here.
It’s just too little, too late.
But Typescript is awesome, keep it up.
Ironically, Google’s competitor (Closure Compiler) has actually been unsuccessful for similar reasons. To this date its main repo is just a clone of the internal master. For whatever reason they’ve never attempted to rebase on open-source release.
So it's maintained in the open on GitHub, it's technically open source in terms of licensing. Yet you claim it's not really open source. Care to clarify?
It’s technically open-source, that’s the point. There’s more to open-source than license. Sorry but there’s no way for me to clarify without just repeating my original comment.
> It’s technically open-source, that’s the point. There’s more to open-source than license. Sorry but there’s no way for me to clarify without just repeating my original comment.
It's free software, and in that sense the license is the only thing that really matters. However, if you're discussing open collaboration styles then that's a whole different discussion. Either your project is free software or it isn't. Whether it has a diverse and open development community is a separate problem, and doesn't fall under "is this project [free software]".
Such as? You seem to have a mental model of things that make a project objectively open source, that don't include the license. I'd be curious what those things are.
I really don’t, it’s more of a feeling. With an open-source release like .NET it seems more like better documentation. In fact that was the case for early commercial Unixes—you needed the source code to actually use the system, but it wasn’t open-source.
Open-source as-documentation (for lack of better term) is still useful. It makes bug fixing a whole lot easier, for one thing. But it’s not quite the same as open-source ecosystem. For that you need to have a diverse set of actors, sharing the same goal. That’s what I think successful open-source project makes. You need to accept the fact that the project is not just yours. Something like that.
Of course Microsoft could do all those things. Who knows, it they’re determined enough they might turn it around. The problem here is like I said Java is just good enough. No one really cares, except people that could use some better documentation, that have been already invested in the ecosystem. That’s why open-sourcing is still valuable, but also why they’ll never gain any adoption of the kind they’d need.
Sorry if that sounds like rambling, it’s sort of late.
They might be open, but there’s democracy and then there’s democracy. See for example recent MSBuild incident (but don’t try to argue about it it’s just an example).
As I said, it’s a feeling. The feeling is it’s Microsoft’s project, everyone else is along for a ride. And that’s fine, but it’s something different. Let’s just not pretend technical merits drive adoption, that’s rarely true.
> They might be open, but there’s democracy and then there’s democracy. See for example recent MSBuild incident (but don’t try to argue about it it’s just an example).
> As I said, it’s a feeling. The feeling is it’s Microsoft’s project, everyone else is along for a ride. And that’s fine, but it’s something different. Let’s just not pretend technical merits drive adoption, that’s rarely true.
Uhm. So many free software projects work like that. A company creates something, releases it as free software. Yes, people contribute (and that's awesome by the way) but in general all of the engineering talent works at the company because they wrote it in the first place (and they hire contributors). At SUSE this is how YaST, zypper, spacewalk, et al development works (with significant contributions from the openSUSE community, but we have teams of people working on those projects so our contributions are more of a concentrated effort). There's nothing new or different about this model of free software development (Sun did the same thing with OpenSolaris and Joyent does the same thing with SmartOS). Yes, GNU and Linux follow the hobbyist model but that's not how all projects work.
i have a hard time with this argument. on one hand what you say is true: C# is a strictly smaller community than java. OTOH that's true of pretty much any language, and yet python, ruby, elixir, swift, golang, etc. communities are healthy and vibrant.
if what you really mean is 'java people won't switch to C# anyway', then i agree, but C# isn't a really a language for them. it's a language for people who don't like and/or aren't forced to use java by their employers.
People who aren’t forced to use Java will choose Scala or other JVM langs, like Kotlin, Ceylon (a favorite of mine) or even Clojure.
C# the language is not exactly that exciting. I get it, it looks attractive next to Java, but it’s still a verbose, corporate-first, sort of thing. If anything, F# is much more competitive. Too bad it’s on CLR.
This has already devolved into opinion territory but I don't think you're giving C# enough credit.
I picked up F# relatively early in it's lifetime (2006ish?), back then there were many language features in F# that you just couldn't do in C#. The gap closed a lot when C# got LINQ, generics, and lambda/first-class functions (these are relatively old language features by now).
If I want to write in an quasi-functional-programming language style I can do it without having the language get in my way. I certainly wouldn't call it a "verbose, corporate-first" language, although the fact that it can be used for that is a bonus.
Don’t forget Rx. Not exactly language feature, but certainly a great contribution to come out of C#/.NET. And who knows if it would have happened without LINQ.
I like the language. Just not enough to use it over JVM. And I think most people feel the same way.
The problem with native desktop Java applications that although Swing is quite powerful, it requires developers to go through books like "Filthy Rich Clients"[0] to really take advantage of it. Which very few do.
To this day I still meet developers that aren't aware how to change the default L&F to native widgets, for example.
Whereas Windows Forms and WPF already provide a much easier out of the box experience, and have tooling like Blend available.
I am curious what the JavaFX sessions at JavaONE will cover.
Sure, it beats Java, but the VM is worse for running other languages, and that's where .NET loses. F# tries to be nice, but reified generics are more of a limitation than help in that world.
In JVM land, now Java just hands you some specific well tested libraries: You write business code with Scala or Clojure, which I'd pick over C# by about as much as I'd pick C# over Java 7.
That said, I have far more faith in Microsoft improving their tooling than I have about Oracle doing the same: It's just that Oracle has to carry a far smaller weight, because the good JVM languages aren't even theirs.
Uhm, the CLR (hence the name) was specifically designed to host different languages that can interoperate with each other and unlike the JVM was not built around the capabilities and limitations of a single language.
> ... if I can't apt-get it, I'm not interested ...
I feel bad for you, then. The Ubuntu repos (and Debian) have almost nothing that is even remotely new and will usually be behind on most things. Trying to keep configs for an ArchLinux box and Ubuntu box synchronized is a bitch if new versions have good features, because you can't use them in your general config unless you give up on apt and install from source.
> This is why I find PowerShell to be an answer to a question nobody asked. If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl to do the heavy lifting.
I'm not sure this is really right though. You could make the same argument against Ruby or Python in favor of Perl or Pike, could you not?
PowerShell syntax is quite minimal, has a novel methodology for strongly typed interactive use, and has the ability to directly invoke managed code. You can write shell components in any .NET language and invoke them (with care at writing time, this can be done very cleanly, but even without said care it's possible with a bit of a mess).
> PowerShell will never accumulate a library as comprehensive as what any of those three have, each has had decades to accumulate packages of all kinds, and more are still being added.
It has the entire .NET library at its disposal. I certainly never feel a lack of support using it.
> This is why I find PowerShell to be an answer to a question nobody asked. If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl to do the heavy lifting.
How are Python/Ruby/Perl going to give you structured objects from "ps"?
That's the promise of object-based pipes. You can get useful records out of your command-line tools without having to write an ad-hoc regex.
I find that to be the issue. You are considering it just RAW text when it is actually formatted text that has been parsable for years with common unix command line tools. It not being in the format you consider a structured object does not mean it's not a object or even parsable. If you are using ad hoc regex I suspect you are not using all the tools available to you.
I feel like Kernighan and Pike do a much better job of explaining than I could ever.
So I wrote a regex to parse this. Seems pretty straightforward, right?
But then I noticed a bug where some symbols were not showing up. And it turns out those symbols look like this:
5898: 00000000001a4d80 0x801058 OBJECT GLOBAL DEFAULT 33 _ZN8tcmalloc6Static9pageheap_E
Notice the difference? Because it's a large symbol, readelf decided to print it starting with "0x" and in hex instead of decimal. I had to update my regex to accommodate this.
That is what makes a parser "ad hoc". You write a parser based on the examples you have seen, but other examples might break your parser. Parsing text robustly is non-trivial.
Worse, it is an unnecessary cognitive burden. Readelf already had this data in a structured format, why does it have to go to text and back? Why do I have to spend mental cycles figuring out which "common unix command-line tools" (and what options) can parse it back into a data structure?
The Unix answer is why are you using a regex on tab-separated values? Wrong tool for the job.
Of course the problem with Unix is that there are a thousand different semi-structured text formats, edge cases, and tools that must be mastered before you can make any sense of it all. Any time you point out the pain a Unix fan can just respond by pointing out your ignorance.
I've actually become a pretty big fan of line terminated json (all string linefeeds, etc are escaped). Each line is a separate JSON object... In this way, it's very easy to parse, handle your use case, and even pipe-through as JSON some more.
In this case, you can have objects as text, and still get plain text streaming with the ability to use just about any language you like for processing.
First off, readelf shouldn't switch between hex and base10. Secondly, that's DSV, so you shouldn't have written a regex for it. You should have either cut, or awk, both tools SPECIFICALLY DESIGNED to do what you want.
DSV: Delimiter Separated values. readelf uses a delimiter matching the regex /\w+/. In AWK, this is $FS by default, so AWK will parse this by default. Or you can pipe
A comparable PowerShell cmdlet would give you one object per line with properties corresponding to the columns. And no, those properties usually have sensible names, instead of "LastColumn".
Of course, for wrapping the native command you'd still have to do text parsing if you want objects. This was more as a comparison of the different worlds here not so much as "if I ran readelf in PowerShell it would get magically different output".
If you wanted the object why did you need readelf in the first place? Why not just read the elf format directly and bypass readelf all together? That seems to be what you are advocating by having readelf passing a object instead of what it does today.
You're asking me why I use a tool instead of parsing a binary format manually? Does that really need explanation?
If that is your attitude, why use any command-line tools ever? Why use "ls" when you can call readdir()? Why use "ps" when you can parse /proc?
You just pointed me to Kernighan and Pike a second ago. I didn't expect I would need to justify why piping standard tools together is better than programming everything manually.
I never said anything about not liking command line tools. In fact I love them and think they do a awesome job!
In any case you just proved my point. You think its insane to parse binary data while scripting and I do too. That is why I think the passing binary objects is insane on the shell.
Now if you were talking about text base objects (not binary ones) then that is an entirely different story and I feel that is what we do today. In your example you have rows which could be called objects, and members which would be separated out in columns. To argue a different text base format is better than another is not something I am interested in doing -- mostly because there are a million different ways one could format the output. If you were to do "objects" I think they would have to be in binary to get any of the benefits one could perceived.
To be honest I feel the output you posted is a bug in readlef. I would expect all data from that column to be in the same base.
I will level with you I can see some benefits of having binary passed between command line programs but I think the harm it would do would outweigh the benefit.
But if you you really wanted to do that you could. There is nothing stopping command line utility makers from outputting a binary or any other formats of text. You don't need shell to make that happen.
What I think everybody is asking for is for command line developers to standardize their output to something parsable -- which I feel that most command line utilities already do that. They give you many different ways to format the data as it is. Some do this better than others, and I think that would hold true even if somebody forced all programs to only produce binary, or json text format when pipped.
This isn't about binary vs text, it is about structured vs. unstructured.
The legacy of UNIX is flat text. Yes it may be expressing some underlying structure, but you can't access that structure unless you write a parser. Writing that parser is error-prone and unnecessary cognitive burden.
PowerShell makes it so the structure of objects is automatically propagated between processes. This is undeniably an improvement.
I'm not saying PowerShell is perfect. From what I understand the objects are some kind of COM or .NET thing, which seems unnecessary to me. JSON or some other structured format would suffice. What matters is that it has structure.
I still don't think you appreciate how fragile your ad hoc parsers are. When things go wrong, you say you "feel" readelf has a bug. What if they disagree with you and they "feel" it is correct? There is no document that says what readelf output promises to do. You're writing parsers based on your expectations, but no one ever promised to meet your expectations. But if the data was in JSON, then there would be a promise that the data follows the JSON spec.
> From what I understand the objects are some kind of COM or .NET thing, which seems unnecessary to me. JSON or some other structured format would suffice.
They are .NET objects, which, in some cases wrap COM or WMI objects. The nice thing about them isn't just properties, though. You can also have methods. E.g. the service objects you get from Get-Service have a Start() and Stop() method; Process objects returned from Get-Process allow you to interact with that process. Basically wherever a .NET class already existed to encapsulate the information, that was used which gets you a lot more functionality than just the data contained in properties.
If the data was in JSON it would promise a that it followed the JSON spec -- but its not, it follows its defined spec, which in the case of readelf is apparently undefined.
Other programs that expect to be machine parsable define in great detail the output. In your initial post I replied to you mentioned ps. In the case of ps it has many options to help you get the data you want without using standard parsing tools. That is because its output was expected to be consumed by both humans and possibility other programs.
Now take readelf on the other hand. It clearly talks about in its man page about being more readable. Its author cares about how it will look on a terminal and even goes through the effort to implement -W which makes it nice to view on larger terminals. It even shows in print_vma, where somebody wen tout of their way to print hex if the number was larger than 99999. If the author really cared about the ability to be parsed they would have added a OUTPUT FORMAT CONTROL section that would provide you the contract you are looking for. Just saying if the data was in JSON does not solve your problem. Why? Because the author of readelf did not spend time to define its output properly in the man page it is not likely he/she would have implemented a json output type when piped little alone take the time to provide the object structures in the man page.
You say it's not about binary vs text but I don't think that can be said. There are lots of things to consider.
* Speed of encoding and decoding.
* Memory consumption issues with larger objects needing to be fully decoded before being able to be used or processed.
* Binary data would need to be encoded and would likely result in much more overhead.
Its not clear to me that a binary option would not be better than a text one. Pipes today are not just used for simple scripts and system management.
There are lots of things that concern me, maybe it is just the implementation details.
* Not all command line programs are written with the expectation to be parsed. How do we handle that? Force the programmer to make all output parsable regardless if they ever intended on the program being used in some script?
* Would a program put the same thing to stdout even if it was flowing to a terminal? Are terminals not for humans?
* Would structure be enforced? One of the awesome things about stdin/stdout is that you can send ANY data you want.
That all said I would love it if programs who intended on their output to be parsed offered a JSON output. I am not against structured output. I am against forcing programmers to shoehorn their output into some format that may not be the best for their program. I think a well designed and documented command line tool that expects to be parsed by other programs will go out of its way to ensure the format is documented and adhered to when operating.
1) Not all data is text. In fact, very little of the data people see/work with day-to-day is raw text. It's silly to transform a PNG image into text to be able to pipe it around. (Or to pipe around its filename instead and have a dozen tools all having to open and re-parse it each time.)
2) There's nothing on PowerShell preventing you from serializing a piece of data to text if you want to. The key is: you don't have to.
3) Systems that depend on 50,000 CLI tools all having their own ad-hoc text parsers are cemented, mummified, cannot change. You can't change the output format of ps (to use an example in this thread) without breaking an unknown number of CLI tools. Even if you come up with a great way to improve the output, doesn't matter, you've still broken everything. This is less (but not none!) of an issue with PowerShell. I like computers to evolve to become better over time, and text-based CLIs are a huge anchor preventing that.
Unfortunately PowerShell relies on everything running on .NET (well, I think COM works, too); the idea of a shell that can expose live objects is useful, but PowerShell's platform limitations in reality doing that make it a far from ideal implementation of that concept. Something built on a protocol that is platform agnostic would be better.
Live objects don’t usually expose any protocols at all.
They only expose an ABI.
Do you know any platform-agnostic OO ABI, besides what’s in .NET?
If you’ll wrap your objects into some platform-agnostic protocol like JSON, you gonna waste enormous amount of CPU time parsing/formatting those streams at the object’s boundaries.
You can run streams of many millions of JSON objects pretty much as fast as the IO can feed it... most of the time, in situations like this, you're constrained by IO speed, not CPU... assuming you are working with a stream that has flow control.
I tend to dump out data structures to line terminated JSON, and it works out really well for streams, can even gz almost transparently. Parse/stringify has never been the bottleneck... it's usually memory (to hold all the objects being processed, unless you block/pushback on the stream), or IO (the feed/source of said stream can't keep up).
Even if printing and parsing is computationally cheap, memory allocation is less so.
If you expose JSON, each serialize/deserialize will produce another instances of objects, with the same data.
The architecture of PowerShell implies commands in the pipeline can process the same instances, without duplicating them.
Another good thing about passing raw objects instead of JSON — live objects can contain stuff expensive or impossible to serialize.
Like an OS handle to an open file.
Sure, with JSON you can pass file names instead, but this means commands in your pipeline need to open/close those files.
Not only this is slower (opening a file requires kernel call, which in turn does various security checks for user’s group membership and file system’s inherited permissions), but can even cause sharing violations errors when two commands in the pipeline try accessing the same file.
> Even if you come up with a great way to improve the output, doesn't matter, you've still broken everything.
You phrase this as if changing things for the sake of changing was a good thing. It is not.
Well, perhaps it is good for the software vendor, but from the customer's point of view, having to re-learn how to do the same stuff over and over every other year is a PITA.
First off, you can pipe binary data around. Most tools just expect text.
Secondly, if you used DSV to parse PS, like you should, adding a new column to the end won't break anything. A fancier parser won't even break if you add to the middle, but that's usually not worth the effort to write.
> What the M$ community fails to see is that text streams can be consumed by __everyone__.
Text can be poorly parsed by everyone, yes. I especially love it when the default tools settings mean two different computers will give different text results, because the installation defaults changed at some point. What's not to love about trying to properly escape and unescape strings via the command line, while simultaneously keeping in mind your own shell's escaping, and having scripts which are neither backwards nor forwards compatible? And this is to say nothing of different distros.
It's as if the text was built for humans instead of my tools half the time, or something. I usually try to centralize my parsing of text in one place so when it invariably breaks, I don't have to rewrite my entire shell script.
Some basic structuring - I hesitate to call it a brittle object model, when most of the time I'm dealing with something more akin to C structs than invariant and abstraction laden Smalltalk or somesuch, or Java's sprawl of factories and inheritance - makes things a bit easier. New fields can be added without breaking my regular expressions. I don't need to worry about escapes to handle special characters. I can trivially dump to text for display (by simply running the command as this is the default behavior of the shell), or to feed into text consuming tools.
It's even more than that: text formatting allows the use of generic filtering and text processing tools. Whereas if you are using objects, you will tend to use less on-the-fly command line composition, and write or reuse more tools dedicated to one particular object or the other. In the end I'm not sure keeping the structuring as the default use case yields an usage so much improved in the real world, because if you work on a broad number of types you will tend to need to know more specialised tools, instead of generic ones, and have less higher-order composition tools at your disposal.
Now of course you can always format to text from your structured object, but this does not matter. What matters is what is convenient in the proposed UI, and what is mainly practiced in the real world because of such convenience and the amplification loop it creates between tools authors and users.
Also some objects are originally handled in text form, and their structured decomposition is extremely complex compared to an informal description and the naturally refinable heuristic which comes with it. For example you can grep into a source code in any language (with more or less efficiency depending on the language, but at least if there is a unique symbol you will find it), whereas trying to get and handle it in a structured way basically means you would need half of a (modular) compiler, and a huge manual to describe the structure, and possibly a non trivial amount of code to actually do the lookup.
The PowerShell approach is not all-bad, though, and obviously there are some usages where it superior to text based shells. But for a day to day usage as a general purpose shell, and a programmer shell, I'll stick to the Unix approach.
Structured data allows the use of generic filtering and structured data processing tools. The basic requirement is reflection, objects being able to tell you about their structural composition.
If code was stored in a structured representation you could still search for a string object containing a symbol name in the structured representation. You can match a structure pattern just like you match a regex to text.
Typical shells can be thought of as REPL for a programming language that makes the file system easily accessible and uses said file system or pipes to pass around data between functions/commands, sometimes as human-readable text. Most programming languages don't encourage passing data around as strings.
> How are Python/Ruby/Perl going to give you structured objects from "ps"?
The usual answer would be "by parsing /proc instead and returning meaningful objects that expose the relevant parts of the OS" but why would you want to do that when you could just output the relevant data in easy to parse XML or JSON?
For the most common use cases there would be a standard function to get structured objects, in Python you have os.listdir() instead of ls, etc. If there is none then you can call ps and parse the result manually or call the system functions with c interop.
For programming, I agree. However, Python doesn't work as a system shell; for that, you want something optimized for running commands, without having to wrap them in a function call with a list of arguments as strings.
I did, and i found it a pretty big chore in practice. Its two modes (shell mode and python mode), never quite knowing which one you're in, I could never quite get used to that. PowerShell doesn't have this problem, because it was designed for the command line first and for scripts second (which is, incidentally, also its major downside if you ask me).
Now that it exists there are users, but when people were hoping for a modern or more portable replacement for DOS batch files I'm not sure this is what they had in mind.
Powershell In Action covers a bit of the history and decision making around powershell; interesting stuff.
Ultimately, given that powershell was initially targeted at systems admins/engineers and that Windows is highly object-oriented(in contrast to Linux's text streams), I believe Powershell was a apt development. Having done plenty of systems engineering across linux and Windows it seems to fit the needs of Windows pretty well; a fantastically unique shell that wouldn't have existed if not for Windows.
> PowerShell to be an answer to a question nobody asked.
It's conceivable that people working in the MS World who have built up (or inherited) a base of PowerShell code are asking, "what would it take to run this on Linux, or at least some of it in some way?"
Of course programmers who don't currently work with the PowerShell at all are probably not asking "I want the PowerShell on Linux" in any significant numbers.
FYI IronRuby never made it into a usable state, never came close to other implementations like JRuby or Rubinius and IronPython hasn't received an update since the end of 2014, which for an open-source project like a language implementation means it's nearly dead.
That said adopting Ruby or Python as a shell language wouldn't make sense. Especially Python with its white space aware syntax, I mean can you imagine piping commands at the command line using Python, as I can't. These languages haven't made it as shell replacements in UNIX either, even though everyone complains about Bash.
Both of these two projects attempted to solve a problem that very few people have - using .NET libraries from Python and Ruby. In exchange, you had to give up all of your existing native libraries. That is a trade-off that very few people wanted. Source: I started the IronRuby project.
I learned this lesson the hard way a few times in my career - always have respect for people's existing code investments.
I can't speak to Ruby, but Microsoft has an active Python group.[1] Granted, it seems to focus on Visual Studio (which I'm told has excellent Python tooling) and Azure.
Neat, but that doesn't seem to be integrated into .NET like IronPython was (well, is, just not maintained it seems). Which is what the post I was responding to was talking about.
I used to love Python when I came to it years about after slogging along through QBasic, VB6, C++ and C, but in this day and age, it doesn't really offer much to me over doing things in modern C#, especially on the CLR. I appreciate some of the changes that were made to the runtime to support IronPython, like dynamic support and the DLR, but if I'm going to be doing something not-C#, then I'd rather go towards F#.
lately i use nodejs and have replaced the cmd tools with node modules. It is more verbose then piping shell commands but easier to understand and the same script can be reused across platforms.
Hehe, I was so happy when I came across a Lua module for Windows administration. And so sad when I found out it was meant for Windows NT 4.0 administration. :(
A current reincarnation of such a module would be really sweet, though.
1: It's the first time I see it, but from a cursory glance and some docs (http://hackage.haskell.org/package/turtle-1.0.0/docs/Turtle-...) I think it doesn't include simply calling external commands (I couldn't find any info on how to wrap an external executable with type, too). Without this, it can't be a real "shell", only a scripting DSL for Haskell. It's not very good for interactive use as it relies on ghci.
2: tcsh is meant for interactive use and is a shell; I used it when working with FreeBSD but never wrote any scripts in it. However, I don't see any mention of types in it's manual (other than file types): http://linux.die.net/man/1/tcsh
PowerShell is statically typed like Turtle, has a nice scripting language like scsh and is meant for interactive use. It's not only typed, but also (partly) object oriented and with full and direct access to all .NET/Mono classes. That also implies that you can write commands in any of the CLR-targeting languages, by the way, like F# or Clojure-CLR.
PowerShell is unique (not literally so, as I'm sure there are other projects doing similar things, but I think it's safe to say that it's unique among widely used shells) in what it provides and in that it takes all that into interactive use. It's definitely worth checking out if you haven't already. Of course, keep in mind its problems, like the mentioned smaller number of already available utilities.
Yes, I skimmed it. But please note that I wrote about "simply" calling external executables, where you can just type the name of a program and have it run. Turtle doesn't even try to allow this (while PowerShell has it), instead going in the opposite direction:
> Most of the commands in this library do not actually invoke an external shell or program. Instead, they indirectly wrap other Haskell libraries that bind to C code.
It's useful for those things that cross the boundaries between a program, and a shell script. I've found once you need functions for a shell program, it's time to step up to scsh.
I also wish scsh ports were more well maintained so people could use their favorite schemes with scsh syntax, but it's quite a large codebase to port, so remember that it's scheme48 scheme.
bits of SCSH have been ported to every scheme out there now. And some have been improved. Irregex, in particular, is a dramatically improved version of the SCSH regex engine.
How do you find dealing with async? Virtually everything I'd want to do with scripting would almost certainly be async in node libraries, and while I'm willing to pay the callback / promise tax in production code, for scripts it seems like the convenience of just dealing with everything synchronously would outweigh the advantage of keeping things in one language.
Personally for me I use es2015 and async/await in shell scripts since it's so close to writing sync code to me. I have an alias to my global babel so it's uber easy to transpile and run the code. I right a lot of node code for work so it's just native to me but I'd recommend others look into it. NPM is such an amazing ecosystem for it.
I agree... though my default reach-for these days tends to be node. I don't much care for ps, even if it's now available on Linux, I just don't see myself using it much... I like C# a lot, and even other .Net languages. But I don't like all the extra typing for one-off scripts, even if they do get re-used.
I can paste various commands for other terminals/environments in a tweet, the same can't always be said for ps. It's just a lot of typing.
As long as you bear in mind that it's main purpose is a usable replacement for the CMD.EXE batch language, then you should be willing to forgive Powershell it's warts.
Except that they added a bunch of new warts that don't have any relation to CMD.EXE. They took the opportunity to re-invent how a shell works from the strongly-typed object perspective but then they copied some of terrible syntax of 40 year old unix shells. I just don't understand it.
That is not quite right. At the end of the day it is all about automation. But we took the position that the way Admins automate is by scripting the things that they do in an interactive shell. Bruce Payette had a great way of saying it - he said that the lifecycle of 99% of scripts starts at the prompt and ends at the carriage return. Then, somethings you want to codify things so you put it into a script. Then as the function becomes more used and more important, you want to make it more and more formal without having to throw everything away and start with a new language.
So YES and interactive shell is important but it is important as a stepping stone to automation.
That is the PowerShell mindset.
Jeffrey Snover [MSFT]
That is not quite right either. I enter about 200 commands on a normal workday and write maybe two scripts a month in a shell language. A shell language should be a shell language, it's not merely a stepping stone to write a script after using a command just once. If that's what I'm looking for I'll use Python. And from the other side, Python would be a terrible shell (by default), and that's fine because that's not what it's used for.
If Powershell is made for both, it has to make compromises both as a shell language and as a scripting language. This is one of the things that annoy me about the Windows ecosystem: it ships with no languages beyond a shell, which makes scripting for it very annoying, and you can't even install a language of choice via a package manager (one has to go find a download somewhere on the web).
This is a really strange argument -- he's saying that PowerShell was designed so that you could more easily move from shell to script to library ...
Your counter is -- you don't do that? Of course you don't, because your shell is bash. That's exactly the point. If your shell is bash, when you write a script you use a different language. What if there was a shell where you could actually script ...
You're absolutely right, PowerShell is made for both, and it has made compromises (like using comparison operators like -eq instead of ==).
> If your shell is bash, when you write a script you use a different language.
The number of bash scripts I see for various -n-x related tasks suggests that this is substantially untrue; people do, in fact, do scripts in bash.
> What if there was a shell where you could actually script ...
I can't think of any shell that you can't script. OTOH, interactive use and scripting are substantially different use cases, so it really makes sense to have both interactive-optimized languages that may be usable in script and scripting languages that may be usable interactively, but focus use of each in their optimized role.
> Your counter is -- you don't do that? Of course you don't, because your shell is bash.
Yeah of course I'd use Bash because Powershell only just became available for other platforms, but that's not my point.
I understood his point like this: the grandparent of his post said powershell has some bad things for scripting. Someone responded to that by saying it's not just for scripting. He said something which I understand as 'that's not quite right, I see a command as a stepping stone to writing a script', To which I finally responded 'I enter a lot of commands but rarely write scripts in [my interpreter of choice's language] because that language just isn't that great, and that's fine because the language is great for its purpose: command line usage'.
And compromises are inherently bad (in a way), that's the definition of a compromise: there are two or more mutually exclusive options and different parties have different first choices. Since they're mutually exclusive, a compromise must be made which is acceptable for both, but it's the first choice of neither (or at most one of them). Compromises are necessary for a dual-use language, but they don't make it prettier for both usages either.
You're correcting Jeffrey Snover about the point of Powershell. I suspect that the guy that invented it and championed it through years of resistance at Microsoft probably knows what is or isn't intended with Powershell.
Besides mirroring the condescending tone, I was mentioning that in my (GNU/Linux) experience -- which is a lot more command-line oriented than Windows -- it's not true what he's saying. I use a lot of commands and very few of them become scripts, thus the conclusion that it's not merely a stepping stone. A command line language can have different features than a good scripting language. Further I was remarking on the limitation of Windows which ships with a "one size must fit all" language: the language used by their shell (and now there's a 2.0 version of that, called powershell).
I'm sure he knows a lot more than me about a lot of powershell-related things. Still, beyond him being (according to you) the inventor and "champion" of powershell, there's nothing that tells me they did UX research into this, so I added my experience because it differs from his viewpoint.
I think you're misreading what was exchanged. That is unless you're arguing with him about the intent behind the design of a product you had no part in creating. He was replying to "The point of PowerShell..." not, "The point of shells...". He was simply correcting someone who misinterpreted the intent.
I think you're also reading tone into his message that's not there. He's been very chill and helpful in this thread even in reply to open hostility.
I'm not sure what "according to you" means, but you can verify it yourself. Feel free to read Wikipedia or you can probably find one of his talks about it on youtube.
After installing a third party bootstrapping program. I suppose that's as close as it gets.
Windows still ships with VBScript, but it's not a language I'd want to look into either. Everyone that has worked with it sighs when it comes up and nobody actually uses it for automation as far as I know (everything's either powershell, good old batch scripts, or custom .NET software).
You could just use WSH (Windows Scripting Host) with either VBScript or JScript the way you've always been able to. AFAIK, Windows has shipped with that scripting solution since like Windows 2000 or possibly even earlier.
I noticed that on old computers (Windows 95, I think, and certainly 2000 and XP) where it would classify .js files as executables. I was about 11-14 so I didn't understand much of Windows at the time, let alone in a foreign language (I'm Dutch) when I wasn't even allowed to use the Internet (dialing up was expensive per-minute stuff), but yeah I remember that.
Still, I've never heard of anyone using Javascript or VBScript to automate anything. It's either powershell (these days), cmd scripts or custom .NET software. I've never even heard anyone mention using javascript in Windows (until apps came around in Windows 8) and vbs is only ever patchwork for legacy third-party software products as far as I've seen in my sysadmin days.
> I've never heard of anyone using Javascript or VBScript to automate anything.
as just one example,
It's common to include JavaScript or VBScript steps in Windows installers. You would never know it, if you ran an installer that used a script to automate a few things.
As one of those enterprise pseudo-sysadmin-scripting-guy, I do agree with the sentiment; and I think the fundamental object-ness of PS was nice and the emphasis on docstrings is nice. However, I still think a python-like syntax would have been miles better. As much as I tried, I just can't get into Powershell -- way too many special characters look awfully "dirty", and $vars give me PHP/Perl-induced PTSD. Deployment is also a bit of a pain.
It works fine as an interactive shell but that's not exactly high praise. It could be better for that and for scripting but instead tries to be more familiar to bash users. As a bash user, I'm terribly happy about it's syntax either but it's the price to pay for 40 years of backwards compatibility. Powershell has no such excuse.
In my admittedly limited experience there seems to be at least 3 aliases for every keyword which makes reading someone else's code a real hassle sometimes. Our ops guys swear by it though.
Yes, this makes for much easier to read code for people down the road :)
I think OPS folks (myself included) sometimes create things thinking nobody but them will use them, then an occasion arises to share, and you're stuck with something functional but horrible to read.
It took just one such occasion for me to convert to writing everything from the perspective that 'other people may need to use this.'
The implicit return semantics are insane, since you can implicitly early-exit when you do things like call ArrayList.Add (which returns a bool to indicate whether the thing was added).
I know the problem you are talking about - it has bitten all of us.
The new language mode you use when writing classes in PowerShell shifts the needle more towards programming semantics than shell semantics and addresses this.
I really like the functional approach of PowerShell. The pipeline, while not the end-all-be-all of perfection, is still very, very good and very flexible. Once you get into the functional mindset, it is very easy to write terse, powerful code.
My real wish is that there was a syntax for writing function decorators and function generators. This would be similar to the [cmdletbinding()] syntax, but would allow me to write my own.
Not if you're in a script and declaring a function. So, no, it's a source of confusing bugs. Implicit return works better in, like, Ruby, where it has to be at the end and can't just randomly exit in the middle of a function declaration.
OK, sorry (there is an explicit "return" keyword that I thought worked the same way). Either way, yes, I know you can avoid it by piping to Out-Null, but that behavior is surprising.
the only place you have an exit from a function, script block or similar is either at the end or at a return statement. Every value that isn't assigned or piped into nirvana gets pushed to the pipeline. I actually can't remember having used return statements more often than once or twice so far (and I use PowerShell daily, heavily).
Then I rarely found the need to use ArrayList. Sure, you can use it, but you can just as well do
$array = @()
$array += $item
A personal pet peeve of mine is actually that lots of people write PowerShell very similar to equivalent VBScript or C# code. And that makes for horrible PowerShell code (in my eyes). The pipeline is immensely powerful and not using it often makes for pretty crappy code.
Most people writing PowerShell scripts are likely dabblers though. Anyway, I have to admit I don't really appreciate the whole pipeline idea; I just know that `$something = Some-Function` sometimes wouldn't do what I thought it would because of these semantics.
Yeah, I wanted to switch to PowerShell but then I had to call an exe with variable arguments from the PowerShell script. That was a quick way back to cmd.
Variables get expanded in the argument list, arguments get quoted automatically based on their content. The only gotchas I can think of are:
& "command with spaces.exe" args
It's a bit unexpected that the & operator is needed.
Arguments sometimes get quoted unexpectedly, or need (single) quotes depending on PowerShell syntax rules to avoid evaluating something.
The worst (but sadly common) thing people can do when faced with problems here, though, is heading for Invoke-Expression, sometimes in combination with all sorts of wrappers, each of them making the problem worse, not better (it cannot get better that way).
The most robust way I tend to use when necessary is to just prepare a single array with arguments and splat that when calling the program:
program.exe @array
No surprises with PowerShell's parsing in command mode because we prepare the array in expression mode. Things are generally more predictable. Automatic quoting of the arguments still happens, though.
(Side note: My own little echoargs replacement is actually a batch file :-)
@echo off
echo.argv[0] = %~0
set cnt=1
:loop
echo.argv[%cnt%] = %~1
set /a cnt+=1
shift
if not [%1]==[] goto loop
)
Is this really so useful in practice? I have never used PowerShell before (nor do I have plans to do so in the near future), but I would imagine the downside is complexity, both for user and especially for developers. With text you just read and parse, while with objects you need to know the object type, its fields,... Whatever happend, I hope text stays here (and I have a bad feeling about it ever since I met systemd).
Could you elaborate on what the actual complexity is, rather than what you imagine?
For a developer writing a tool, exposing an interface as objects is far easier than having to constantly serialize data into a string. Keep in mind that once you serialize data into a string, you can never change the order of the data or else other tools will break. Not so with a object based interface.
In other system-admin type tasks PowerShell hides complexity. For e.g. finding the IP address of a network interface is simply
As someone who recently got his first exposure to PowerShell, I'll chime in that the case I needed to deal with was one where it added complexity.
In summary:
I need to pull data from a company we're partnered with, and get it into our systems. They ship it as SQL Server backup files. OK, no big deal, we can mount those up and run queries to dump stuff out to whatever format we like and ingest it.
Except part of that data is files, which were stored in the DB as binary blobs (in a table along with filenames and metadata). Need to get those out of there and into our storage as real files. OK, no big deal, I can just query for filename and file contents, dump out onto disk and then batch-transfer into our archival storage, right?
Except no matter what kind of obvious shell-idiom method I try to use, it doesn't work because PowerShell stubbornly insists that I must be working with text data when I'm using all those handy shell constructs, and so the files end up corrupt.
So now instead of just being able to shove data through pipes like I would on Unix (which, for all its faults, doesn't really care whether I'm sending binary or text data), I have to have my script go and instantiate objects for creating a file and reading and writing binary streams, and remember to clean up after them, and my script ends up way more complicated.
What would have been a pretty simple script anywhere else now has to know a bunch of .NET libraries and how to create and work with their types, and that's less useful to me.
Its true that powershell being a new tech probably doesn't have the large library of "known ways" to do things that the UNIX world has. And BTW I'm not even a powershell person. I write embedded software in C++ for a living ! Having said that, I'd imagine you would parameterize something like
Invoke-Sqlcmd -Query "SELECT ..." and write it out to a file?
or avoid it and just use BCP "select blah" queryout "c:\file.jpg" etc etc
To be honest, I don't consider this to be that bad, I was just replying to a person who thought that bash was "simple", and powershell was "complex". Also, I don't have a problem with either tool. I'm very open to the idea that Microsoft can and does create awesome tech, despite having other shitty products.
You guys are just proving my point. _I_ already know how to do it using bash. Its just that using objects and querying properties is far simpler. And I just took a random example because its a common thing that someone would want to do.
You really couldn't have asked for a better chain to demonstrate the validity of your point. Having been blinded by my own world view and biases in the past I sympathize with them but this is too funny not to have a little laugh at their expense.
I think what others might be demonstrated is that there isn't a reliance on what object Microsoft decides to ship to you, in *nix we can combine commands and builtins and get the information in the structure we need without having to wait 8 months for a system reboot (or worse a paid for upgrade) that will give us what we need.
Well, to be fair, what was demonstrated here was that people didn't actually extract the data after multiple edits and tries, regardless of what tools they had at their disposal.
I think the idea of an object pipeline is sound for precisely this reason, but I don't think it should be connected to any one language or platform.
First of all you are incorrect in your assumptions. You are reliant on the author of the tool in both cases. Just as there is no "UNIX Person" who decrees that all executables on UNIX must output text, there are vendors other than Microsoft who write software for Windows. You can write one too, it takes about 5 minutes to expose a powershell interface. And I'm not even a .NET developer. I write embedded software for a living.
It is parsing text, but that text does have some consistent structure. Contained within it are properties. One can make the same comment of reading JSON or XML but I'll admit those are more like what you'd choose for an API. In general there is a not-entirely-obvious (to me) question of where does "formatted for machines" end, and "formatted for humans" begin? We can't say "it's for machines if and only if it must conform a schema that's defined in advance", not any more.
>I can only use this feature by using a .NET language.
It goes without saying that a binary interface requires use of a system that understands that particular binary interface. With .NET being open source though, anyone is free to write a bridge to using the objects in COBOL if they wanted to.
>I can't take the strongly typed objects into a Node.js script, or a perl script.
Yes, because they have no concept of strongly typed objects. You can convert objects to strings if you're using an incompatible tool/language. I don't know what your point is. Sorry..
The point is that this thing is less valuable by being limited to only being used by things that Microsoft created (in reality). Text, with all of its flaws, can be used by everything.
I guess the point is the point. Potential bridges that do not exist already and probably never will do not matter -- extending that idea of potentiality I could say that you are free to reimplement the CLR in Fortran, so PowerShell is compatible with Fortran? Oups sorry you already reached that ridiculous point, only with Cobol.
Virtually any language can already input and output text. That's the real lingua franca of computing (even if it is messy and dirty), not .Net objects...
>Potential bridges that do not exist already and probably never will do not matter
The difference between impossible and possible is important. Please read what I was replying to. Using your argument, Anytime a project is open sourced, you could reply with "who cares about potential since nobody has actually done anything with the source". Good job.
>Virtually any language can already input and output text.
Um, you seemed to have missed the obvious point that objects can also be converted to text through powershell if the other side can only accept text. So, I don't quite know what you're complaining about. Could you detail your actual technical complaints with powershell, rather than having a philosophical argument?
>That's the real lingua franca of computing (even if it is messy and dirty), not .Net objects...
I don't agree with your opinion, nor with the idea that only a two-choice system can exist.
If you prefer shorter (and similarly obscure if you don't know the language):
ls -file|? LastWriteTime -lt ((date)+'-2')|rm
I actually find PowerShell's version more readable, even if more verbose. But code is written only once and read a lot of times. I may not know what the +2 actually means with the -mtime argument. How would I change it to look for the age of two hours instead of days?
Just as an aside (and this probably applies to powershell as well), what if you have mapped folders from multiple servers into your file system and those servers are in different timezones? Something like
> With text you just read and parse, while with objects you need to know the object type, its fields
parsing is actually a huge pain in general. a lot of unixy tools produce different output when their stdout is not a terminal to be more pipeline-friendly. That output is generally easier to parse (often tab-separated or so), but you still need to know the same things about the output (the "type" could be whether it came from `ps` or from `ls`, and the fields are just in some static order that was usually chosen decades ago, and if it's extensible it's cumbersome).
If `ps`, `ls`, etc produced typed output, I would be sooo happy.
When I worked at google I thought about what it would be like to have a unix userland where all the parts communicate with protobufs instead of raw text. There are some problems (like portability of message definitions, which was a solved problem within google, but which is harder when you have multiple parties/entities/companies contributing message definitions).
I wrote a bash script to sort processes by age on an old redhat 4 machine.
What sounded like a very simple thing, quickly ended up being a complete nightmare of bash, python, and an unholy bunch of linux commands.
It was ugly, and a terrible way of doing it, but we got it working.
It would have been amazing if i could pipe PS into something that could sort by the dates on the 4th column, and return the last column in the sorted order.
Pretty sure the section in the `ps` man page called OBSOLETE SORT KEYS indicates that even the `ps` on very old redhat machines could do this out of box... see the field `start_time`.
Failing that, consider the `etimes` field (seconds since process was started) used in conjunction with the -o feature (control output format) so you can put `etimes` up front then simple pipe the output to `sort`.
Those things being said, the situation you describe is indeed hairy. `ps` output was designed for human consumption, that's why the process age field format varies (1w vs 10:15) and that's presumably why you wanted to bring python into the mix!
...You might have had an easier time looking in /proc...
Indeed, this looks promising:
cat /proc/[0-9]*/stat|sort -n|less -S
EDIT: hm, I don't know how old /proc is!
EDIT #2: But try doing ANY of this on an ancient Windows machine! The "peer" of your old Redhat machine would be what, a Win2k3 machine, or a Win2k? How old is PowerShell? :)
Honestly it was a quick and dirty one-time project, and I just needed to cobble something together.
I was more just commenting on the fact that with a "strongly-typed" shell some of this stuff becomes much easier without every single tool needing to add all these options.
It gets much more "Unix Philosophy" when every command doesn't need it's own sorting logic, and doesn't need it's own date display logic, and doesn't need it's own way of displaying it to the user.
It can output an object, which can be piped into another program to use what it needs which can be piped to something else to do some sorting which can pipe it out to another to display it in a nice table.
I hear you. I myself was impressed by some things PowerShell can do easily.
But let's not throw out the baby with the bathwater... The unix shell ecosystem is powerful magic. There are LOTS of things that can be done there with great ease.
Not to mention--and I understand full well that this stance is falling out of fashion--MICROSOFT ... KILLED... my PAPPY! :|
Yeah i ended up using awk, some python for the date parsing and sorting, back to bash for grep and another thing i can't remember, then into the program that actually needed the data.
It was a mess, and I remember thinking it would be such a simple thing that was so much more complicated than i thought it would be.
mysql > SELECT column_name, data_type, numeric_precision FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name LIKE "%int%"
The reason I chose that particular example was not for the database integration (which is very cool) but for the fact that it's actually something I did.
Point is, imagine trying to grep anything tabular in bash. Hypothetically, the output of ls -la for files with execute permission that have been modified in the last day. Or, more realistically, anything involving ps.
When boxed in-between the fearsome foursome--`bash`, `sed`, `awk`, and `emacs`--every problem involving tabular data (2D) is trivial. Hm, I'll give a shout-out to `sort` and `uniq`, too! When you go 3D, you might need to bring in `join`.
But anyway, you'd use
find ~/bin /usr/bin -mtime 1 -executable -type f -ls
..to provide `ls -la` output for every executable, ordinary files (not directories, which are often +x) modified in the last 24 hours, under the directories /usr/bin or the bin/ under your home directory.
Most users do not find awk trivial. Most people basically use it for {print $colnum} when simple columns are available.
PowerShell's integrated query syntax that you can patch into various things is a lot more consistent and reasonable. It is difficult to imagine volunteering people into a world of awk, sed and cut with good intentions in one's heart.
I think this speaks to the difference in mindset I've seen a lot in this thread. It basically comes down to people who have gained and honed experience with these tools vs. people that haven't. Yes, for someone with a lot of experience with these tools something like Powershell is redundant... for them.
Some people just want to get shit done because this is just a footnote in a task that's a small part of a project. Utilizing gained proficiency in several tools is cool but if you have a choice between doing that and just taking advantage of a simple tool that streamlines things and lets you move on to your next task quickly the correct choice is obvious.
It could be that these tools are better and may be more valuable in some situations but asking someone to spend time gaining proficiency or mastery when another path is available is just premature optimization.
I have no time for tool worship, language worship, and other sentiment like that. These are tools, not people. I'll use the best I can buy, borrow, or make until I see another better one comes along.
If that is true, then it is equally true of the authors of shells.
But of course, it's not true. Many great and lasting tools have been founded out of corporate work in a bid to improve mindshare and ease recruiting by increasing the esteem of corporate engineering efforts. Many of the things we think of as "open source" and "free" were built subsidized either directly or indirectly by capitalistic interest.
But I'm not concerned about keeping it. Ossifying my process so that I can give myself the luxury of time of from learning is not something I'm prepared to do, because I think it will make me soft and less capable as an engineer. I regularly rotate my editors and learn new languages because I don't want to become like the sad specters of the previous generation of software engineers I see now, starched shirts and ties and a fanatical devotion to the dated technologies they stomach in order to collect huge consulting fees from corporations.
I've switched off Fish to Powershell as my primary shell for awhile. It's interesting.
And paste for tasks requiring the opposite of cut, then there's tr, the lightweight sed. And jq for structured queries on JSON. But, all these are built in to poweshell, essentially.
Er, it's a world all of us live in. Say, for example, ‘ps’ or ‘ls -1l’.
Then you end up using awk, which is rather like a half-assed version of powershell to work around shortcomings with the unix everything-is-a-string design.
AWK isn't a workaround. It's a tool for processing tabular data, in a textual format. The fact that it can solve so many problems is actually a strength of the EIAS design.
And those programs usually emit proper CSV. You could use regex to match on what you want, but AWK was literally built for this. It's fantastic, it works great, and if you don't use that tool that was built to solve your problem, that's your own lookout.
Well, yeah, but I mean the concept (grepping in tabular data).
If, hypothetically, bash had database integration, ls Columns would probably return something like
foo varchar 30 NOT NULL
bar decimal 8
baz int 4
Which would be a royal PITA to search for anything in—if I were to search for ‘int’ I'd get all int columns and any columns with int in the name….
I could've used an example involving process lists, but I didn't want to trigger PTSD in anyone who's had to grep the output of ‘ps’ for anything non-trivial. (Plus, the database thing was something I actually did a few days ago.)
awk is rather similar in many ways to powershell, actually, but untyped. I agree most of what powershell does you can do with awk, but it's often less pleasant than powershell (in my experience).
More specifically, you can find out all of the type information about anything you get access to by piping it to 'get-member'
get-netadapter | get-member
Which gives the following output http://pastebin.com/bM0cBeXb about the type information for the result of 'get-netadapter'. In addition to this, tab completion for properties is available.
Alternately, you can get all of the current properties of any existing object by piping to 'select' (to get a general subset) or 'select * ' to get all properties. Note that this gives all information for the objects in the pipeline.
get-netadapter | select
get-netadapter | select *
The first will provide a short list of fields for all network adapters, and the second will provide all properties.
'Select' can also be used to easily filter the properties of items in the pipeline to ones you care about.
get-netadapter | select Name, MacAddress
You can also just generally filter the pipeline based on generic filters (give me all of the network adapters that are currently not enabled, and have a driver provided by Mellanox).
And ultimately, if you want to use powershell to do the stuff that typed objects are good at, and then leverage command line tools to do the rest, you can if you like.
Which will take all of the network adapters, gather their name and mac address, and spit it out to stdout as CSV which can then be sent to any number of external tools, or using the '-expandproperty' flag to 'select' can give just a bare list of properties one on each line.
What you can't do so easily with text is also just auto-convert that object directly to JSON or YAML or CSV, for that matter, and forward that easily into your favorite JSON/YAML/CSV exploration tool.
Ok, that sounds good. But what if some developer decided that some field would have prefix "ABC" in all values, then the value I am interested in, then some value I am not interested in at the end? (you say this can't happen? Clue: WMI.) Can I take the value, convert it to string, make some magic on it and convert it to another object entirely?
I don't know, I never had any problems with text parsing. I love strong types in programming languages, but in system administration I think it would only get in the way. But as I said, never tried it, so I might be missing something.
The default regex functionality is absolutely terrible (maybe this can now be fixed with a PR). You'd write your own cmdlet[1]. A few things are demonstrated in the cmdlet: functions that take pipes, functions that return pipes, anonymous functions, actions to perform before processing a pipe and dynamic objects. The Powershell syntax gets ridiculed for being ugly, but it's beautifully expressive.
> Can I take the value, convert it to string, make some magic on it and convert it to another object entirely?
Replicate the above without using [Regex]:: (i.e. use -matches and $matches). It forces you to radically depart from a typical Powershell mindset and approach.
I'm not certain what you're getting at; -match doesn't handle multiple matches so you can't replicate the above just with it? Yes, that's a bit annoying. You could match line by line:
> "But what if some developer decided that some field would have prefix "ABC" in all values, then the value I am interested in, then some value I am not interested in at the end?"
I don't see why this would be a problem for PowerShell. Just use Select-String to strip away the characters you're not interested in. The advantages that derive from passing around objects aren't based on all data being automatically in the format you want.
There are many ways to accomplish this (in general). My favorite is that you can create 'computed' properties on the fly which means you can continue to keep the strongly typed object in the pipeline, and still have access to the data you care about.
You use the exact same functions (first, last, substring, replace, etc) as you would normally. Then the rest of the pipeline has it available if needed.
Yep! There's a few ways to accomplish such a thing, but probably the most straight forward would be the Foreach-Object command (aliased as "%") to do a mapping from source object to something else. Free form objects/dictionaries are easy to make and consume.
Yes, it's quite useful in practice, because the alternative is knowing exactly the format of text some command is going to produce and munging it, which is hardly better
It's nice when it works, but it's a pain to write a command-line utility that outputs into objects. Whereas in Unix, everything automatically outputs into text, so you have to work hard to prevent it from being chainable to the system.
There are also details of Powershell that are very annoying, such as the lack of < redirect, and the difficulties it has on binary output sometimes.
I've done some Powershell work in my brief time as a Windows sys-admin. I personally struggled with it. The object interaction can be complex and the naming feels less scriptable and more directly out of the C# world.
That being said, I can see the appeal and I know people who love it. I have seen some amazing shit written in it, like this security tool that was just open sourced this year:
I have used Mono/GTK# for years on Linux and prefer it over other GTK wrappers. Powershell might be worth a try on Linux, but it will take time to build up the libraries and wrapper needed to interface with most of the Linux subsystems (unless Microsoft as ported a lot of useful stuff in this initial release).
People don't (or shouldn't) use bash/zsh/fish for really complicated stuff anyway. I have a feeling most sys admins/devops people will continue to use Python/Ruby for complicated tasks; and they both have a ton of libraries that help avoid the stdin/out/parsing situation you have with chaining together regular shell scripts.
I really hope this drives a new wave of experimentation and development in shell interaction. Text as the lowest common denominator will never go away but having more powerful constructs available will be great for a lot of use cases.
I wonder what's the difference between an object-pipable shell and a lisp machine (only 10% troll here). Kalman Reti said that passing pointers as IPC meant zero perf penalty btw. Text serialization has design value sometimes, but the amount of cpu cycles allocated on string munging seems flabbergasting.
> Once you've piped strongly-typed objects around between your shell commands, text scraping seems barbaric by comparison
This is why Powershell is a great thing on Windows. Windows works with .Net objects and Powershell works directly with those objects.
It's also why I don't see the reason to be excited that it can run on OS X and Linux now, because neither of those systems work by passing .Net objects around. You parse text with bash/sed/awk/perl/python ... because *NIX works on text based config files and commands that output text.
No... It just means that all features may not be available. It makes no comments on how well the existing features work and how compatible they are with the Windows version.
I mean, `Get-ComputerRestorePoint` and `Get-WmiObject` don't mean anything on Linux. I assume those won't be ported. `Get-Process` is certainly meaningful; I assume that will be. The ports might also have cmdlets not available on Windows. I could certainly see cmdlets to interact with `apt` in Ubuntu, or `yum` in RHEL. Especially since PowerShell is focused on provisioning and managing machines.
"can't use all that power for much, since the pool of PS-compatible tools and utilities is much shallower than it is for Unix shells"
The explanations are obvious though... aren't they. Text might be barbaric, but every programming language in the world makes sending text to stdout dead easy. The unix shell utilities are smart to support that case.
Elixir's REPL (iex) makes a surprisingly-good environment for "pip[ing] strongly-typed objects around between your shell commands." It could use a form with added convenience magic, the way Pry is for Ruby, but I've managed to do some decent interactive ops work on systems through an Erlang-remote-shell (remsh) connection to an IEx shell.
IIRC correctly, the major PowerShell push started with Win Server 2008, and headless server mode (no desktop). A lot of WinX sysadmins pissed their pants that they couldn't use GUI's to control everything, and PowerShell was pushed as the way to do server management w/o GUI's (crazy, I know).
It's junk. Tried to get something useful done and version incompatibilites and poor documentation made it a losing battle. Type theory is over-wrought for getting things done. My boss doesn't care that I have to cast an integer to a string.
I'm hoping this means running EF migration scripts on Linux will be less of a nightmare. It was one of the biggest issues I ran into developing a Web application in .net hosted on Linux.
I take a super utilitarian view. At the end of the day, PowerShell on Linux is another tool in the toolchest.
PowerShell has a POV about abstracting gorp into high level task oriented abstractions, object pipelines, structured data, and the workflow from interactive shells to ad hoc scripts to formal scripts to production scripting.
That is what a number of people are looking for in their tools and that is why they like PowerShell.
We are constantly looking for ways to improve PowerShell and make it more useful so I'd encourage you to suspend disbelieve long enough to kick the tires. If you see some stuff you don't like, I would be very interested in hearing the details. You don't need to be polite in your feedback but I'd ask you to be specific so that I can identify potential changes.
If you are happy with your existing tools - happy days!
But no one uses them, and no one is building an ecosystem around them. That's what I'm hoping will change thanks to this announcement. Even if it's not PowerShell itself, if a different strongly-typed shell gets traction because of this, I will be happy. That's what I was trying to say with my comment.
I've been using bash for 20 years, trained a few hundred RHCEs, and spent a large part of my life working for Red Hat and then IBM's dedicated Linux group. So yeah maybe I don't know any better, but that's because I haven't encountered it before. Piping objects to 'where' and 'select' is simply a better approach than scraping text. The implementation details - having to use .net to make cmdlets - aren't great but the fundamental idea of powershell is excellent.
We've banned this account for repeatedly violating the guidelines. If you'd like to commit to only commenting civilly and substantively, you can email hn@ycombinator.com and we'll happily unban the account if we believe you'll do so.
Back in the day I thought VAX/VMS was amazing. Particularly things like file versioning, however, I had to leave it behind and do things on UNIX (and then Linux) terminals. That change was relatively painless even if I did delete a file or two that I should not have. However, I have moved on and nowadays I would not have a clue how to use VMS beyond typing 'dir'.
If the VMS 'shell' was suddenly available on Linux, despite its 'proven' benefits in various scenarios, I just would not have the working knowledge to get going with it again. Furthermore, there would be no easy answers on StackOverflow to do basic things such as recursively renaming part of all filenames, or batch modifying the content of said files.
The problem PowerShell has for me is that it might as well be a souped up version of VMS (or even BBC Basic). It is not that I am adverse to learning, it is just that I left Windows World a long time ago. Much like my move from VMS to UNIX, there were some good things that I left behind (such as the file versioning thing), however, if you move to a new way of working then there is no need for the old stuff. I appreciate that PowerShell is new, but it does not suit my way of working, i.e. cribbing answers from StackOverflow and tutorial guides.
>We will be extending the PowerShell Remoting Protocol (MS-PSRP) to use OpenSSH as a native transport. Users will have the option to use SSH or WINRM as a transport.
PowerShell is interesting, and I don't consider it evil. I know some people swear by it, and I am not one of those people - I'll take my AWK with a side of Ruby/Python, and a helping of SCSH, thank you very much.
It does seem a bit out of place in a UNIX system, though. In Windows, PowerShell ties to a ton of strongly typed databases (SysReg, &c) and windows APIs together into a nice package. But on UNIXes, those APIs are already well exposed elsewhere, and that data doesn't exist. UNIX data is typically stored as text, or can be extracted as such easily. We don't tend to do a lot of binary object munging. I mean, maybe if it integrated with databases, it could at least be handly for dealing with RDBMS scripting, but those bindings are available elsewhere - ruby, perl, python, etc. And MS could build bindings to POSIX, so you'd get a PowerShell version of tools like PS that returned objects, but what are you going to do if you want to munge /etc/fstab, and, say, generate user accounts for all your boot filesystems? I could do that with a line of bash and a few lines of AWK. I don't know why I'd want to, but I could.
So to conclude, PowerShell seems like a neat tool, but it was designed for an ecosystem that doesn't exist on the *nixes.
> But on UNIXes, those APIs are already well exposed elsewhere, and that data doesn't exist.
I'd go with exposed, but not necessarily well. Examples I run into all the time: get an IP from an interface, change iptables entries, even change fstab options. Sure, if you write enough regex, it will probably work most of the time. But it's never going to be as reliable as get_ips('eth0'), or get_filesystem('/tmp').options.add('nosuid').
It's unreliable, because you don't know where you'll find 'inet ' or what's the specific spacing of the output. Also, that's not the ip, because you've still got the cidr attached.
And no, fstab is not easy either. For example it fails for:
# /tmp has option_blah in order to ...
changing it to:
# /tmp has option_blah,nosuid in order to ...
Confusing the next person to look at the well intended comment.
It's no impossible to implement fairly reliably, it's not going to take days to do, but you're likely make mistakes. And in both cases you're spawning extra processes instead of just getting the information you need.
PS. also slightly annoying is that everyone is so concerned about the text processing part that barely anyone checks what the tool itself can do. (ip -4 -o a s dev eno1 | awk '{print $4}')
>PS. also slightly annoying is that everyone is so concerned about the text processing part that barely anyone checks what the tool itself can do. (ip -4 -o a s dev eno1 | awk '{print $4}')
Actually, I spend a while digging through the manpage to try and find this. ip has awful docs.
Yes it will. If you mean the $2 part, the comment character takes the place of $1. Or did you mean something else? (either way, I tested it before posting)
...Which is why I said it was ugly. The grep probably won't break, the cut is more likely to. This is in part because the 'ip' command isn't very well designed. A better solution I found is
ip addr show wlan0 | grep -Po 'inet \K[\d.]+'
Which is less likely to break. If your system supports it,
hostname -I
Works great. However, this seems to be an ubuntuism, and it doesn't work on my system.
Are you out of your mind? Or do you just not understand what you're saying? Adding an interface to PS to get this info natively is good. But Lennart has a magic ability to get everything wrong, and he's NOT messing with my fstab any more than he already has.
>Then which audio system is simple to set up AND allows per-program volume, effects, etc?
JACK
>Which init system "just works" without requiring me to copy and modify a hacky init script from stackoverflow?
s6, runit, bsdinit (yes, really).
>Lennarts products are controversial, but they’re the classical "least worst" options.
No, they're ugly, broken messes that look nice. Systemd is a monster. If it was an init system, I wouldn't complain, but it's not. It's an init system that's also a cron replacement, a syslog replacement, an automount replacement, a docker replacement, a consolekit replacement, steadily gaining a tight integration with udev, overcomplicates everything it touches (we need a hostname daemon now?), provides libraries to high-level userland that are DELIBERATELY POSIX incompatible, promotes binary formats for no reason, flies in the face of tradition for no reason and breaking code (screen sessions die now), overcomplicates PID1, increasing the chances of a kernel panic, and that's just the tip of the iceberg.
Systemd is a tiny step forward, followed by a massive step back, and of you can't see that, you haven't been paying attention.
Try getting it to run with basically anything, seriously. Pulseaudio is plug-and-play.
> Which init system "just works"
Neither of the ones you mention fit that description
> It's an init system that's also a cron replacement
Wrong.
SystemD is a project with a similar branding issue that KDE has.
Systemd is a project.
It develops a set of libraries.
And, on top of that, an init system.
And a cron replacement.
And a syslog replacement.
You do not need to use them together – just like you can use Gtk apps in a KDE environment – but together everything works best.
> for no reason and breaking code (screen sessions die now)
For the reason that shutting down my PC means it spends 10 minutes waiting for screen sessions to kill themselves, and I can not stop that without either doing a killall, alt-print-reisub, or just pulling the plug? It also provides a way to keep things running still.
> increasing the chances of a kernel panic
Never had one, and I’m running bleeding-edge arch
> Systemd is a tiny step forward, followed by a massive step back, and of you can't see that, you haven't been paying attention.
Systemd is the least worst option.
There is not much of an alternative for such a kind of project, that properly handles things in a more modern way.
I want a userland that works plug-and-play, I want syslog as a daemon with database, not as a fucking flat file. I want more dbus, and, ideally, I’d have all config files in a database or registry for easier administration and sync.
I want proper udev and containerization, I want programs of my user to be killed upon logout unless specified otherwise.
Lennarts projects don’t provide what the old crowd of nerds want.
But they provide what the average user want, and what’s required to make Linux actually usable for your sister, your grandmother, and your retarded neighbor that can’t even turn on a PC.
It also reduces the workload for server admins, like me, again.
I can write systemd unit files in a few minutes, can easily stream syslog over the network from multiple servers and aggregate, I can easily handle namespacing of software.
>I can write systemd unit files in a few minutes, can easily stream syslog over the network from multiple servers and aggregate, I can easily handle namespacing of software.
You could do 2 of those three things already (syslog and namespacing), and bsdinit init files aren't hard to right.
>Systemd is a project.
>It develops a set of libraries.
>And, on top of that, an init system.
>And a cron replacement.
>And a syslog replacement.
>You do not need to use them together – just like you can use Gtk apps in a KDE environment – but together everything works best.
That's BS. A lot of it's all one executable.
>I want a userland that works plug-and-play, I want syslog as a daemon with database, not as a fucking flat file. I want more dbus, and, ideally, I’d have all config files in a database or registry for easier administration and sync.
Well, if you want Windows, you know where to find it. But stop trying to turn my UNIX into it.
>I want programs of my user to be killed upon logout unless specified otherwise.
Well, you're in an EXTREME minority.
>But they provide what the average user want, and what’s required to make Linux actually usable for your sister, your grandmother, and your retarded neighbor that can’t even turn on a PC.
...and if you think "the year of linux on the desktop" is ever coming, you're delusional. Besides, Ubuntu had a reasonable UX for that audience long before systemd came around.
>I can write systemd unit files in a few minutes, can easily stream syslog over the network from multiple servers and aggregate, I can easily handle namespacing of software.
Software namespacing doesn't require systemd. Syslog streaming has been a thing for a long time. And BSDinit, s6-rc, runit, openrc, sheperd, nosh, systemXVI, perp, watchman, and a variety of other init systems exist that make writing unit files easy. Many of them predate systemd.
That’s not even true if you use the packages for debian or redhat. In both cases, it’s over a dozen packages for systemds independent projects, and you can replace each of them seperately, if you wish to.
I’ve done so in the past on a server where I couldn’t update everything because it was an old-style containerized VPS with shared kernel.
> Well, if you want Windows, you know where to find it. But stop trying to turn my UNIX into it.
> Well, you're in an EXTREME minority.
Then go and use UNIX, aka BSD. This is the linux world, and most people want it – as seen in the countless times when distro maintainers held a vote pro or con systemd.
> ...and if you think "the year of linux on the desktop" is ever coming, you're delusional. Besides, Ubuntu had a reasonable UX for that audience long before systemd came around.
Does Ubuntu allow me to configure all services and systems via a neat UI? No? Why? Because it still uses a flawed file-based approach, which is useless in the 21st century.
Ubuntu replaced user-facing UX, but even there not everything could be properly done yet.
Despite that, Ubuntu also created their own init system, display server, syslog daemon, and a registry for configs.
So, they had the same stuff systemd now provides, and more.
> You could do 2 of those three things already (syslog and namespacing), and bsdinit init files aren't hard to right.
> Software namespacing doesn't require systemd. Syslog streaming has been a thing for a long time. And BSDinit, s6-rc, runit, openrc, sheperd, nosh, systemXVI, perp, watchman, and a variety of other init systems exist that make writing unit files easy. Many of them predate systemd.
I’ve tried unit files for other init systems, it was a pain in the ass.
Systemd is the first where every software provides matching unit files, I can easily write my own, and it just works.
Again, the entire point is that it just works and is still easily configurable and modularized.
It's not modular, it's a mess. Systemd may be organized into separte components, but make no mistake - it's a monolith in sheep's clothing.
As for unit files, nosh, at least, provides the same format as systemd. s6 "unit files" are just the command to daemonize the process, in most cases. BSDinit and OpenRC are pretty simple as well.
>Then go and use UNIX, aka BSD. This is the linux world, and most people want it – as seen in the countless times when distro maintainers held a vote pro or con systemd.
Linux is a UNIX too, and most of the people who know and care about init systems don't want it, judging by the massive protests that show up constantly from people who understand the technical aspects.
>Again, the entire point is that it just works and is still easily configurable and modularized.
It "just works," but it's a technically deficient mess that goes against everything unix is about, is a monolithic beast in disguise, and is trying to force everyone to use it.
If it weren't for that last bit, I really wouldn't care.
>Systemd is to sysvinit what Linux was to Hurd and other microkernels
That's just not true. Linux was a return to a simple, tried-and-true model from a more complicated and theoretically beneficial one. Systemd is a devergance from a simple, tried-and-true model to a more complicated one, that really doesn't have many benefits.
>And "s6 "unit files" are just the command to daemonize the process, in most cases" is a horrible solution, too.
>The reason systemd unit files are so awesome is because they’re simple configuration, not having to string complicated commands together.
So are S6 unit files. So are a lot of config formats, actually. Sysvinit was really bad, but other inits had moved on. The complicated commands are only required when the app had complex requirements, or when init is stuck using fragile pid handling, which S6 and the like don't need, because they implement proper process monitoring.
>You fundamentalists had 2 decades to make a working system
BSDinit, OpenRC, s6, hell, even upstart, and countless others are all working systems.
Anyways, I'm not a fundamentalist. I don't think that systemd's unit files are broken. What I think IS broken is systemd itself.
Init has a few jobs: managing startup, managing shutdown, reaping processes, and handling daemons, etc. By this point, init's job is well defined. How it should do them has been argued, but the jobs are well defined. That is what init should do. NOTHING ELSE.
Systemd tries to do everything. Thus, it's overly complex, AND it's a single point of failure.
>It’s time to end this.
I quite agree: When Lennart stops making me try to use his piece of trash disguised as software, I'll stop complaining about it.
Great! So I can now run bash on windows (with WSL), open tmux, and open powershell prompts in each tab: the loop is complete!
I wish powershell's documentation was a bit more clear about what's powershell 1.0, 2.0, 3.0... Etc.
A lot of time I'd find a verbose and tedious way to do one thing because the code was written in "old powershell", when the latest version had a much more elegant solution (either because of new language features or simply more cmdlets were added, removing the need to fall back to some C# syntax)
On stack overflow it often means read more than the chosen answer, somebody would have added a newer way to do it. I wish Microsoft "hey script guy" blog had some versioning info (this post was last updated in 2016, using powershell 3, see powershell 2 version here)
The official cmdlet documentation is versioned, though for some reason it only seems to have versions 3, 4, and 5. Which, agreed is very annoying, since Windows 7 has version 2 by default.
Does tmux work well on Windows? Everything Microsoft has been doing recently with bash on Windows has made me consider buying a surface pro as my next machine.
I have Surface Pro 4 i5 and while I personally love it- I can't really recommend it due to various issues (battery life is shit, sleep is bugridden so best to hibernate only). It's uncomfortable as a tablet, and near useless for the main thing ultra light computers are used - traveling a lot.
I use it at home. The pros are a really tiny size, yet it packs sufficient computing power for many things, awesome screen. I can use it at a desk or grab it to the sofa - but I need to keep the power adapter handy.
If the execution was as flawless as Apple's Ipads are it would probably be the best computer ever (by my subjective scoring which appreciates a pen and sRGB precise screen probably way more than the average user).
But as it is - and I'm comparing it to my Ipad pro which makes all the problems stick out like a sore thumb - it's not an obvious recommendation.
It really is annoying that the battery issues are so persistent. That's the main thing that has kept me from even going to a store to check one out.
I've been stalking /r/surface and I've been seeing complaints about this for over 6 months. Apparently it's more Intel's fault than Microsoft's, but still very disappointing. I love the form factor and I want to support Microsoft lately.
I was a big GNU Screen user on linux, which unfortunately doesn't work on WSL. So I tried tmux, and it works fine for the few minutes I tried it:
Create new tabs, detach session, etc.
But once you close the Bash primary window, the session dies, so it's not as persistent as running tmux on a Linux machine, which makes sense given how WSL works. I haven't figured out yet if there is a way to always keep WSL running in the background so that I could close the bash terminal, reopen, and reattach my tmux session.
Which is why I have no doubt the Universal Windows Platform will become the Universal Platform and be a true desktop/tablet/mobile/server/etc Java competitor on all major OS (mobile, traditional and server).
It must be rather frustrating for Microsoft to look back a decade or more and see they were sitting on the exact technology a hell of a lot of people said would be their future had they made it cross-platform from the start. I remember using C# for the first time around 2004 and thinking "god damn if this were cross-platform like Java it would kill Sun". Back then Microsoft couldn't see past the tip of their nose though.
Yup, I've been playing around with C# more lately and it's such a good language.
Things like embedding with native code is trivial compared to Java. Want to call a delegate(with extras) from a C function pointer? Sure thing, just specify some syntax and away you go. Marshalling strings? Just works.
Totally a breath of fresh air compared to the mess that is JNI.
Me too, back in the first .com wave I had access to early betas and was quite critic of it, nowadays I enjoy using it more than Java.
Although I do still like Java, just don't like having to wait for Java 10 for some of the .NET features already available (possible free AOT compiler, value types, reified generics, JNI replacement).
IMO this is why Google is currently building a new universal OS from scratch. They know the power Microsoft can yield with this. It will take some time to catch on, but when it does, it's going to be so big.
I think this can be explained by the fact that Microsoft has a long history of trying to kill OSS software and alternatives to it's virtual monopoly in the desktop OS market, killing innovation across many technology sectors in the process. Being cross-platform was the opposite of their goals. They specifically created their own version of open technologies and promoted them as the best/only way of doing things on windows - see directx/opengl, and mictrosoft intentionally degrading opengl performance on windows in order to hurt the OSS community.
Microsoft operates as though, and under the assumption that their software is needed. It's a faulty premise. The presumption is arrogance incarnate.
As a business I can fully, completely operate without any of the their software, profitably, peaceably, and with at least the same amount of friction or less. And more securely.
If you think otherwise, you are ignorant of the tools available for free and of some intelligently thought-out engineering.
Quality of a language is such a trite, over-used discussion that the results/goals of software are completely forgotten. This too is arrogant ignorance.
How many software engineers are there in the world? It's more than taxi-cab drivers. More than Dr.'s More than teachers. Cooks, farmers, and builders (not combined of course). This is evidence of software for software's sake. There is no quality. Just a moving shell-game of "you need this".
I love writing algorithms--that PowerShell and .NET are some kind of something that will solve people's problems any better than something else is simply marketing. I have a pet rock to sell you.
Maybe consider moving away from Windows/ You are living in their world. If they want PowerShell to be adopted their whole program management will make sure you can only get things reasonably done with PowerShell. It's their world, and you don't have to be a non-paid employee of Microsoft.
The only related revenue stream M$ seems to have left is Azure with some vague notion that giving away their ecosystem will drive people to use their cloud more
Microsoft does not grow food, build houses, create clothes, cars, or protect you from foreign invaders. The basic human needs are not met by MS. The business that claimed that open-source was a cancer is just a marketing company with memetic feedback loops. Plenty of over Operating Systems are out there. Their office product is no longer relevant (vis. a vis. Libre Office, Google Docs, and others). Are you telling me a multi-billion dollar company cannot compete with a startup such as Google? There's a reason why. They do not create. Just marketing.
>Their office product is no longer relevant (vis. a vis. Libre Office, Google Docs, and others).
You have no idea just how wrong you are. Excel has no competitor worthy of even talking about. Libre Office is a mess. Google docs doesn't do everything word can and is in the realm of a toy web app.
As a full-time Linux user, I have to say that bash is overrated.
For instance, on Windows I could press the up arrow through my history 20x to find a group of commands I want to run, press Enter, then press DOWN for the following command, then enter, then DOWN until I run them all. In Bash, I need to press UP 20x every single time.
Yes, you can customize it a lot, and you can script it but default bash isn't that special to me.
You can use control-O for that, which does "execute this command as if I pressed enter, then when it's done bring up the following command in command line prompt". So if you have a set of commands to run that are 20 lines up in history, you can first find them (with 20x UP or by reverse-search), then just control-O control-O control-O control-O ... to execute them one after another.
Don't forget control-R. I find that even the GP's example of up arrow 20x to be ridiculous when I can type C-r and a few unique letters that were anywhere in the command to find it instantly.
I am not the GP but I learned about C-o by reading the man page. If you spend a lot of time in Bash, why not learn to use it properly? There are less than ~100 key bindings, see "Readline Command Names" in http://linux.die.net/man/1/bash You should at the very least skim over them.
operate-and-get-next (C-o)
Accept the current line for execution and fetch the next line relative to the current line from the history for editing. Any argument is ignored.
Maybe it went something like "god damn it why can't I ctrl+v to copy, what other ctrl keys are there..."? And now I'm looking at http://ss64.com/bash/syntax-keyboard.html and being amazed I didn't know about / forgot a few of these. It'd be nice to put some of the good ones on a coffee mug, I first learned useful vim commands from a coffee mug...
Nope, because I've never used control-keystrokes to do cut-n-paste. I always use (and love) X-style mouse cut-n-paste and hate that there are no good implementations of it on other OSes.
Dating myself, the way I learned copy and paste was CTRL-insert and SHIFT-insert, and those still work fine in konsole, mate-terminal, most any other half-decent terminal program, and straight text mode outside of X.
This turns out to be because on OSX the terminal driver defaults to throwing away control-O which is rather unhelpful of it. "stty discard undef" fixes it.
> For instance, on Windows I could press the up arrow through my history 20x to find a group of commands I want to run, press Enter, then press DOWN for the following command, then enter, then DOWN until I run them all. In Bash, I need to press UP 20x every single time.
I think the Bash default is better here. If I wanted to run 20 closely related commands, I would make them a single line; far more commonly I want one command from 20 commands ago and then the last command, which is a pain to do on Windows. Much worse is that Windows seems to lose all history every time I close and reopen a command window.
you can start typing a previous command and go up through your history for all commands starting with whatever you currently have typed into the bash shell. Reset the scrolling through the history with `ctrl+c`.
Personally, I hate that powershell remembers your position in the shell history. I don't want to maintain a mental model of the shell history rollodex and where I am in it. That's one less part of my brain that I can't use for thinking about the code. If I have a set of commands to type over and over:
history 20 | head -10 | sed -r 's/^ *[0-9]+ *//g;' > new_script.bash
The search by previous characters seems to be the default behaviour of FreeBSD's t/csh. Confused the hell out of me at first because Ctrl+R isn't reverse search.
Ah. I use the standard emac'esque commands to move over by word. alt+b for back word, alt+f for forward word. Not at all intuitive... windows line movement commands win over emacs bindings form an intuitive perspective.
For the record - the PowerShell team loves Bash and much of PowerShell is modelled after it.
To be fair - we also modelled portions after VMS DCL, Perl, TCL and AS400 CL.
There are lots of awesome engineers out there working on different platforms and we learned a lot from of them.
Jeffrey Snover [MSFT]
In many bash configurations, ctrl-o is bound to 'operate-and-get-next’. If you hit up 20x, you should then be able to hit ctrl-o to run that command, and your history will automatically go to the next command down. Check it out, it’s great!
I don't see a tremendous amount of accumulated wisdom in Powershell. It may be newer, but I'm not sure they looked at prior art or had design members who were shell gurus in any other shell than `cmd.exe`, which is a horrific shell. Why are commands horrendously long in powershell? If you spend time in the shell, you don't want your fingers falling off due to overuse. Aside... one of the design decisions Microsoft took that absolutely blows my mind is using `\` for file hierarchies instead of `/` when you're writing an operating system in C!
There are lots of aliases for cmdlets for exactly this reason - interactive use. The verbose commands are useful when reading a script that isn't working at 3am in the morning when your boss is breathing down your neck to fix it "stat!".
RE - your aside '\' vs '/' - I think most of us still shake our head at that decision.
I don't know if that was really a "decision" or an unfortunate compromise. DOS inherited /foo style command line parameters from CP/M, and DOS 1 didn't have a hierarchical file system, so there was no conflict with the path delimiter character UNIX was using. When DOS 2 introduced directories (and brought over the cd, md, rd, etc. commands from UNIX) suddenly we had to choose between massively breaking backwards compatibility or using something else, like \, as a delimiter.
> Aside... one of the design decisions Microsoft took that absolutely blows my mind is using `\` for file hierarchies instead of `/`
That decision goes back to MS-DOS 2.11, circa 1983. (DOS 1.0 didn't have subdirectories.) The model for DOS directories came from UNIX, which at that time was the very aggressively defended intellectual property of AT&T. As Microsoft was an AT&T licensee -- for Xenix, their version of AT&T System 7 UNIX, which was Microsoft's vision of the future of operating systems when PCs became powerful enough: they licensed it in 1978 and resold it via OEMs -- I speculate that they were afraid that AT&T might sue them for patent or copyright violation relating to the filesystem design if they went with a forward slash.
`/` is not a first class citizen. You encounter different oddities throughout the windows environment in the shell and external utilities when you use `/`.
There are many (*nix-inspired) aliases for frequently used commands though. I think PowerShell's preference for verbosity is a strength more than a deficit. Why "grep" when select-string "sls" is so much more intuitive and promotes readability.
Cmdlets tell the shell their parameters, and their parameters are long names, so you can just tab through them to find a) the available parameters and b) the ones which might help you.
Well, it's not an improvement, it's just a different design-choice. I for example much prefer Bash's way of doing it.
Also, Bash does have a way of doing it the Powershell-way, if you want that, as others have commented.
Which is also why I disagree with your statement. Bash may be old, but neither the underlying technology nor the use-case changed much in that time. So, there is not a whole lot where you can innovate with a new tool, at least not for such basic tasks, and instead, Bash has matured throughout 27 years to perfectly fit that underlying technology and use-case that millions of people have.
It seems weird that the feature of PS that you're most impressed with is something so... trivial? You have full access to your history in both shells so you could easily emulate that feature in bash/zsh or you could create a special binding to run all commands from a point in your history so you only have to press Enter once.
As someone who has created templated subclasses with both private and public virtual inheritance, I don't think that's a bad thing. C++ is wildly powerful, but it's a mess, and most people should stick to easily understood idioms/patterns, for the sake of maintenance if nothing else.
I also highly recommend enabling the pgdn/pgup bindings for history-search-forward/backward. If you are searching for a prefix it's a bit easier than ^R.
>For instance, on Windows I could press the up arrow through my history 20x to find a group of commands I want to run, press Enter, then press DOWN for the following command, then enter, then DOWN until I run them all. In Bash, I need to press UP 20x every single time.
I know other users have addressed this issue in other comments with solutions, but you do know this isn't an issue with Bash so much as it is with readline, right? There are plenty of ways to customize readline/inputrc to make it do what you want. There's even linenoise, if you hate readline for what it is. It wouldn't matter how good Bash got as a shell language, this scenario wouldn't be any easier unless readline also gets a facelift.
I was about to refute you with a recent example of mine that just needed to serially execute a bunch of commands, and it's well over 20 lines. Then I remembered that I needed to introduce some branching logic, said "not in bash, I'm not", and rewrote it in Python.
So for me, it's not a matter of LoC but complexity. Need to copy file from here to there, then commit to git repo, blah, blah, blah, then kick off a build? Whether it's a 1000 lines or 2, I don't have a problem with doing that in bash. But as soon as I need some logic (or on rare occasions, perf), I go to something else.
Given that, why do you cut shell scripts off at 20 lines? I ask in the spirit of being open to the possibility that maybe I'm doing it wrong. :-)
Well it isn't a hard limit. I am okay with a 25 line script for example ;)
Well really you are more right than me. It is about complexity more than the number of lines. However for the most part the longer something is the more complex it is to work with even if it does something simple. Managing a 1000 line bash script is almost certainly going to be more painful than doing it "properly" in Python or Java or whatever.
It mostly comes to do this for me "is this thing going to be something I need to understand and manage outside of my own little bubble?" if it is then I want it to be better designed/maintained and I find that gets easier as I work "down the chain".
It mostly comes to do this for me "is this thing going to be something I need to understand and manage outside of my own little bubble?" if it is then I want it to be better designed/maintained.
Thanks for taking the time to reply. And you're right, even I wouldn't want to maintain 1000 lines of serially-executed commands, meaning I have a line limit somewhere as well. So maybe a mix of complexity, and at some point length even if I don't put a number on it like you did.
Everyone has their own way of working :) I first got into things with batch files on Windows 95 and quickly realised how awful they are when more than 5 lines! I quickly moved on to their Windows Scripting Host stuff which was better but still painful. Then VB6 then C++/MFC.... The usual Windows user story.
Scripting on Unix is a lot better but it is only recently that places seriously use VCS for scripts in my experience so before the days of Git if you wanted to do anything with an iota of manageability it meant you did it in a "real" language hehe.
That's cool, we are just different is all. I use Python for automation tasks that require more than what I can do with Bash/Batch/PowerShell but don't warrant a full write/compile/deploy process. If your job is working in Python then sure you are going to use it way past the limits of what I would use it for.
Things are a lot nicer these days than 10 years ago though with PyCharm, etc. it makes managing a Python project so much easier.
Me too. And personally I'm somewhat conflicted as to whether or not it's a good idea. On the one hand I love python and use it for virtually all code I write (that doesn't have to run in a browser) and I'm quite convinced that for what I'm doing it's the most efficient language for me to use. On the other hand I have found myself on more than one occasion really wishing python wasn't quite so dynamic when working on large python code bases and if I was starting a new large desktop application today I'm not sure python would be my first choice. There really is something to be said for a better/stricter type system.
When my shell program gets longer than a few lines, it's time to break it into some makefile targets with intermediate files.
I have in the past written a moderate size program in Powershell (installer for a program which needed a lot of configuration for a lot of components and got installed into a lot of different environments). And while that probably wasn't the best approach in hindsight, it wasn't actually that awful an experience.
It's not a million miles off writing a moderate size JavaScript program, except that it has an actual module system.
I wish they hadn't chosen dynamic scope though, nor the silly array result squashing behaviour.
Also the IIS 7 Powershell Provider - that was bad.
It's just default behavior and its customizable for a reason. Different people have different preferences, powershell's default behavior is in no way 'better' than bash's. You can also use a different shell like zsh.
It is just another tool in the toolchest.
As you know, on Linux, there are tons of great interactive shells and it is sortof a lifestyle choice.
You might want to kick the tires and see whether it fits your lifestyle or not.
If you use Bash as your shell, think of PowerShell as you would python or perl - something you write scripts in and call from Bash.
Anyone who's done any kind of if else while for in bash/zsh and powershell knows what a relief PowerShell is. I'd love for PowerShell (PSReadline actually) to support the common options of readline- e.g. HISTIGNORE, HISTCONTROL etc
PowerShell is interpreted, as far as I know. Class declarations may be compiled on the fly, but I've never used them; I'm basically still at PowerShell v3/v4.
It is about Microsoft being a neutral player in a multiple platform world. Microsoft want people using their tools as it means they are more likely to use their platforms and services. Be it PowerShell on Linux or Bash on Windows or VSCode on macOS they want to make the best tools in the hope that it will push companies/developers towards Azure/SQL/Cloud as that is where the future for MS is.
Windows is on life support and they know this. The future isn't about companies buying hundreds to thousands of Windows licenses but an instance of an OS which is billed per X/month. They have seen businesses are more than happy to enter into such a model with Office 365 and The Cloud so it makes sense for them to move in that direction.
How is Windows "on life support"? Desktop/laptop computers are not going anywhere, at least until they invent some kind of VR (and no, mobile OSes like iOS are not suitable for doing real work and managing data in a networked environment the way desktop OSes are). Windows still has at least 90% of the OS market. People and businesses could change to an alternative, but they just aren't, and they aren't going to unless something really drastic happens to force them to (and this would likely have to include many popular applications also providing support for one or more of the alternatives). In the business/enterprise world at least, there is simply no indication that Windows is in any danger at all of losing marketshare or revenue, in fact it's probably going to increase since businesses seem to love the software-as-a-service model.
MS could probably just make Windows free for home users (and make even money on them with advertising and selling telemetry data and also support calls), maybe small business users too, and then soak the big businesses with high fees for their site licenses and SaaS and other services.
Windows revenue has been shrinking for a few years now. The growth is in Azure and related cloud technologies.
Sure Windows probably won't ever go away (until a massive paradigm shift as you mention) but businesses are not upgrading like they once were. You have a mixture of reasons for this, BYOD has chipped away at some of it and forced companies to allow non-Microsoft (i.e. Apple) systems to connect to the network for remote and onsite.
With more and more work being done in the browser it makes specialised software that forces upgrades to Windows out of the picture.
I have worked for a number of large companies and recently the reduction of annual spend on Windows clients is pretty staggering. With almost all "managers" going BYOD with their (mostly) MacBook Pro's and onsite machines lasting a good few years longer than they used to it means we just don't need to buy as much from Microsoft.
You can see Microsoft know whats up as well. Windows Enterprise subscriptions and Azure are the future for Windows in business. For home users Windows is already "dead" from a revenue perspective with the exception of charging via the OEM at first sale. Android, macOS, iOS, tvOS, all the operating systems home users are aware of are free upgrades on supported devices. This is where Microsoft has a problem as obviously they support everything so they can't lock out some users with a "2011 Dell XPS". However going forward I expect we will see them do something along these lines with limitations around what minimum hardware is supported so you will only get upgraded to the latest Windows if you have AVX2 or such based hardware.
I would agree that Windows is nowhere near being on life support but I do think that the size of the desktop OS market is shrinking and that in the future we may see a lot more home computer users move to tablet only or something like a Chromebook.
Enterprise users are much more slow to move but as enterprise SaaS offerings become more robust we may see a lot more businesses move towards a thin-client (a la ChromeOS) and SaaS model to meet their needs. Microsoft is doing the right thing by quickly getting out in front with Office 365. If they hadn't maybe something like Quip or Google Docs could have starting stealing significant market shared but now I don't think many enterprises would bother switching if they can stay with Microsoft.
When PowerShell was written we knew more than when Bash, let alone the original Bourne Shell was designed. Powershell understands that parsing is sometimes hard. Powershell does not invite data-insertion attacks with the the fervour of Bash. In short, it does many things right that would Bash do too if it could be rewritten.
And yet -- I've rarely found it useful in practice. But it's hard to judge that, since I don't do as much in Windows as I do on Unix. So this move will level the playing field a bit, and we shall have a fairer comparison.
I think PowerShell would actually be way ahead of Bash. But we have things so much better than Bash. Python/Ruby let you do a lot of amazing stuff (not shells, I know, but still immensely more useful scripting languages than what you can do in Bash).
Also, if you're still using Bash, I'd highly recommend looking at newer shells. Fish is pretty amazing. I've been using it over over a year. There are others too like Zsh.
I actually use Zsh most of the time, but my point is that pretty much every Linux box has Bash at this point, so I'm a bit curious what Powershell has to offer.
Powershell is really more about the scripting language and the command processing. The "shell" part is modular. On Windows you can still use most cmd tools inside powershell and with its commands.
So you shouldn't give up bash, you should use both and take advantage of the tools that help you get the most done.
Wow. I've been very impressed with Microsoft lately. They could have easily set themselves up as the next IBM, in the sense of being a monolithic corporation relying on corporate practices, but they've been spewing some crazy fun things that shows they're not out of the league yet.
Might just be MY miscontrued perception that microsoft was becoming old/monolithic, but really excited with the direction they are heading.
Microsoft have been hugely focused on hiring people who have experience outside the Windows bubble, and have been encouraging cross-pollination of ideas with the OSS world. There are holdouts for the old guard of course (there's still a BU that requires Windows Phones), but I think everyone knows that the Microsoft of old cannot compete today by bringing an attitude that pleasing the CFO is all that matters. There are techies in the boardroom now who are familiar with the early-mover advantages of free software and the importance of the OSS culture to their workers.
So when a company of massive resources, who can see the writing on the wall, decides to hire thousands of people who actively disbelieve in the value of monolithic lock-in and who enjoy using open-source tools, and empowers them to build interesting things without regard to company loyalty, the result is Powershell on Linux and Bash on Windows. Oh, and a whole lot of goodwill amongst the people you had previously alienated.
Let's hope the awesomeness they are showing on the developer relations side finally leaks into the Windows side and they stop their shenanigans with W10 and Android and VFAT patent threats. It's so disappointing to see this kind of split-brained behavior.
Developers are pretty much the only demographic that might realistically switch operating systems, so they only really need to cater to them. The rest, they can pretty much treat however they want, and they won't lose them as customers.
Exactly. The developers they can get to work for them are "useful idiots", whose work they then use to further screw over their main customers, who aren't going to abandon them no matter what, and also bring in more developers to their platform and away from the competition. As long as these developers are easily and naively wooed by a bunch of BS talk about "the new Microsoft", they get to help MS with their evil shenanigans.
Sigh. I don't even know how to start with this one. I'm a "useful idiot" because I'm interested in making PowerShell work as a proper Debian package, or because I might look into what's required to get it into FreeBSD?
Seriously?
MS is not a monolith, it's made up of lots of different people with different ideologies and different goals.
I think the parent comment's point, with which you are likely to disagree but haven't fully addressed, is that even the best intentions of developers can only serve the ends of the less noble segments of Microsoft.
In response to the flagged sibling comment: I use Linux as my main OS and have almost non-stop since ages ago. I have in the past half-seriously wondered whether infamous projects that make Linux worse were MS false flag ops. But here, rather than making personal accusations (regardless of how true they might seem to you), it's probably more productive to provide a counterexample, a reminder of MS's negative behavior, or evidence that they haven't really changed. Then either people can see for themselves, or people within MS will see they still have a long way to go and keep pushing for change from within.
I agree with you! Even if this is all for microsoft's bottom-line (and there's nothing wrong with that since they are a business after all), we all (devs and consumers alike, directly or indirectly) stand to benefit. The prospects certainly seem exciting.
I'm glad that Microsoft is doing stuff like this. But, I can't imagine a lot of Linux users are going to want to use it. As a former Windows developer I always found Power Shell to be lacking in nearly everything I wanted to do on the command line, in fact, I mainly did shell commands on a remote Ubuntu server.
I think the most succinct way of describing it is that UNIX tooling is scatterbrained and at times a little ugly, but each command exits because it solved someone's real-life problem. PS by comparison is a set of tools that try to be all encompassing and solve problems MS devs think people will have.
Our PS repository is full of scripts that do nothing but abstract PS into useful high-level actions.
PowerShell's object-passing approach ultimately is a heck of a lot more composable and reusable than the "blobs of text with AWK and SED magic" approach that its predecessors took.
It's somewhat surprising to see people here arguing that the pure text methodology is superior. It's difficult for me to imagine a technically motivated scenario that is improved by having everything dump out inconsistently formatted and specialized plain text.
Great example, ifconfig and netstat. Regular interfaces and an object model over these under the covers is a whole universe better than the ad hoc tabular formats they present.
Finally! I love the methodology behind PowerShell. It's more verbose than bash, and that's OK to me. The readability and _consistency_ far outweigh the extra characters. Now, I get that coupled with all the flexibility of *nix tooling.
It's going to be a long road to hook everything up, but this has the chance to finally standardize command parameters.
I'm sorry. This is truly amazing to me and I'm quite excited!
So, that's kind of the beauty to me. For long lived scripts I can always type out `ForEach-Object { ... }`. When I'm ad-hocing there's always `%{ ... }`.
Bash is a strange beauty, but part of that beauty is how arcane it can be. Generally, people use the smallest parameter possible in scripts instead of the full name. While this can definitely continue with PS, the community has decided on "alias for ADHOC and Full-Name for scripts". This helps the readability tons if someone hasn't memorized all the command parameters.
It's happy to gobble up whatever improperly-capitalized cmdlets you throw at it, heck you can even forget some letters ;). And with a little tap on that tab key it will helpfully fix your laziness so there's no trace of it in your shell history.
I'm not a huge fan of PowerShell but you're not really making a good comparison here. Bash is "just a shell" that executes commands with a few bits of syntactic sugar here and there to make it easier to string those commands together and perform some primitive logic.
PowerShell is basically an interpreter that works more like a Java Virtual Machine. Consider the hypothetical example that you need to lookup 100 users in an LDAP directory from Bash: You run the openldap command 100 times in a loop. That's 100 processes that need to fork and quit in a serial fashion in order to perform the task at hand.
To do the same thing in PowerShell you'll be calling a native PowerShell LDAP API which will execute all 100 lookups without having to fork any separate process resulting in a vastly faster execution.
It's better to think of PowerShell like the Python or Ruby interpreters. IMHO, PowerShell is a vastly superior improvement to cmd.exe as a shell but on a Unix host it will be inferior to bash for day-to-day tasks.
I fully agree -- they're two different animals. IMHO, it's a symptom of the two different paradigms of running a process on Windows vs Unix.
I worry about getting out of the habit when scripting on Linux of forking off a new process for each iteration of a loop, rather than pushing the loop into powershell, for better execution.
Using java + jython, results in 15 pids, 2010M Virt / 196M Res.
Jython is probably the more apt comparison, vs the compiled interpreter. But, it falls into the same trap as powershell: You couldn't invoke jython in a loop like you would 'python' or 'perl'.
I'm not so sure it results in a vastly faster execution, at least on OSes which don't need half an hour to spawn a new process. So, yes, of course it was needed for Windows, because there creating a new process is so expensive that sometimes you've got to mortgage your house to do it.
I was going to cover this topic (the cost of starting/forking a process) in my original comment but decided it would be a bit tangential since I was able to make my point without it.
On Windows the cost of forking a process is high whereas on Unix/Linux it's low but on both platforms every application has a nontrivial startup time that will be significantly greater than a native call within an existing process.
Looked at from another direction, the unix world works beautifully with slim VMs, unlike windows which requires gigabytes of ram. There are plenty of 0.5GB ram unix VMs out there that do their job just fine... and to fork over 20% of that ram just to do shell scripting? Not very portable.
ConvertTo-CSV converts each property into a single string. Rich object? Tough. You get a string. If the type does not support a nice string representation, you get type names, like "System.Collections.Generic.List`1[System.String]" in your CSV output.
ConvertTo-Json serializes in-depth ... properties that are actually rich objects become nested dictionaries instead of simple strings. For instance, the "Parent" which is a Folder object. Recursively, up to the depth allowed.
Anyway, when you convert back, you don't get actual FileInfo and DirectoryInfo objects, you get dynamic property bags. So those complex properties remain dictionaries ... which ConvertTo-CSV turns into strings in a very different way.
If you specify -Depth 1 when you ConvertTo-Json ... the resulting output would be very similar.
I don't have a powershell handy, but one thing that might be causing changes is that json dictionaries usually don't have a defined order (parsing libraries are free to render the keys of a dictionary in any order); are the changes you're seeing just ordering changes?
ls and grep are aliases in powershell, so not exactly the same, but pretty close
For your first question: you have converted types.
When you did a ConvertTo-Json, you serialized the System.IO.DirectoryInfo type into JSON compatible data. When you ConvertFrom-Json you have a PCSCustomObject.
The conversion to CSV is different for the System.IO.DirectoryInfo than the PSCustomObject.
This has to do with the sub-types (converting from objects into serializable JSON objects). It also appears to handle time strings differently as well.
ls is an alias for Get-ChildItem. However, grep is the native command in this case. While such things work to some extent, it has the problem of turning your objects into plain strings. Effectively the same as if you had run
(ls | % Name) -match '_'
The fact that the file's name is its default string conversion can be pretty annoying when dealing with results, e.g. from gci -recurse where it's quiet important to retain the path to the file. So, careful with such things. On Stack Overflow I tend to scold people who reduce PowerShell objects to strings without good reason ;-)
Powershell is the first language where I managed to save time by writing a program for a task. Needless to say, I'm thrilled at the prospect of using it on Linux. The only thing that I hope they will need to work on is startup time making sure it performs well on low-spec servers (It can be a bit slow starting cold on older laptops).
I am not sure what your example does. How does it differ from
find . -name ".docx"
ls -Recurse | ? { $_.Name -match ".+docx" }
And, if these are the same, why type in twice as many characters as a command? Actually,
ls -R | grep "\.docx$"
is, I think, closer in spirit.
Now, the issue is things further in the pipeline, I imagine.
The find solution allows selection on type name, type, date, depth, mounts, properties, file system type, ownership, relative age, inode, symbolic link. find actions include delete, exec, print, quit. This can pass on needed information to further pipeline commands.
However, an interactive shell is meant to allow expression of commands with a reasonable minimum of keyboarding (thus ls and not dir). I could argue that the "Unix" command should be
find . -n ".docx"
(because the most common -n... option used is -name).
I think, to be fully reasonable the find command you want
is
find . -regex ".*\.docx$"
which is still shorter than the PS example. I am not sure
that for most interactive tasks, PS is a win. It does appear
to be more elegant, but only usage over time will show if
it is indeed more productive. I am looking for academic productive studies to be done, now that both bash and PS are available on common platforms.
If you're just talking about ls/dir, then PowerShell isn't really a win. The advantages are clearer when you start talking about lots and lots of commands. A lot (but not all!) Unix tools have flags to control their output format or input filters, but that means you have to memorize different flags for all of them. OTOH, in PS you just pipe to where/sort/select and they work for everything. IMO PowerShell has better separation of concerns.
I've been hoping for a fully-featured Linux version of Powershell for a while, very awesome. I've used various Linux distributions for a long time and find bash very practical usually, but when I started learning Powershell when working in Windows environments I've been impressed with how easy it was to do some powerful stuff, and felt conflicted since I wanted to be able to use similar functionality and scripts across Linux/Windows.
Powershell isn't competing with bash, it's competing with Python. So then, how does it compare against Python? While plenty of small scripts are still done with bash, I think Python is pretty common for serious production needs. I use it for virtually all scripting these days.
I had a recent experience where a developer spent a day and couldn't get Powershell's webrequest library to do what he wanted. I was able to accomplish the goal in 15 minutes with Python on a Debian system.
Sure, there are different ways to do various things, especially depending on what one is familiar with and what the task at hand is. It sounds more like you knew how to get something done with your tools faster than the other guy/gal in this case. :)
There are obviously many factors such as domain knowledge and language familiarity. I respect the developer who failed to do this in Powershell, he's very capable.
In this case, I would chalk it up to Python having a longer history and being more mature. Neither of us were particularly familiar with this domain, but because of the quality of Python's online community and documentation, it wasn't difficult to accomplish the goal.
Powershell really became interesting for me recently. I was working on diagnosing a production issue where one of our win32 applications failed on loading a .net assembly (dll). The error messages weren't terribly useful, and I wasn't getting a good stack trace. While Googling around, I found out that you can randomly instantiate .net objects from the PS command line. A few commands later and I had a full stack trace indicating the problem.
It's been such a difficult road. The consistency that PowerShell provides makes discovering things simple. There's no referencing a MAN page to see which parameter switch it is for _this_ tool. Yet, there's been no power. No ability to harness OSS tools in that way. Now, maybe I can have both.
So some reason, the rpm that Microsoft provides doesn't list all the dependencies that it actually requires. What is even worse, the compiled binaries inside, require quite outdated versions of icu & openssl that are not available in Fedora 24 any more.
$ find /opt/microsoft/powershell -name \*.so -type f | xargs ldd 2>/dev/null | grep not\ found
libcrypto.so.1.0.0 => not found
libicuuc.so.50 => not found
libicui18n.so.50 => not found
(libcypto* is openssl-libs package, libicu* is libicu.)
The build scripts for PowerShell right now require an existing powershell binary to work. If I can untangle this and work out what's required to build from scratch, then that'll probably help you build a proper Fedora RPM.
(I'm looking into it for Debian, but the work will help other distros, too).
If people could be relied upon to maintain their own systems and apply security and other updates, this wouldn't be a problem. The unfortunate reality, is that they can't.
PowerShell is an improvement on Bash, but I'll still be using F# .fsx files for my .NET scripting needs. That said, the real reason I'm excited about this release is that there's finally a great example of a very large project based on .NET Core [1]! Most other examples I'd seen were pretty small, so this is really helpful.
PowerShell is one of Microsoft's better offerings, it runs rings around Batch, thank heaven. However, it has its own quirks. Loads and loads of stuff done implicitly (unwrapping arrays as parameters when constructing an object?) and the .Net integration can be annoyingly flaky. Add in the bizarre syntax, some kind of halfway house between Bash and C#, and you quickly end up writing enough code to hang yourself.
My favourite: defining functions more or less like any other C-inspired language:
function abc([string]$param1, [string]$param2)
{
#do stuff
}
...but if you CALL that like any other C-inspired language:
abc("string1", "string2)
PowerShell mashes the two strings together and ignores the second parameter. Drove me mad when I started out. Incredibly inconsistent.
I like Microsoft's effort here, and the design is a lot better, working with objects, but not on my Devuan box!!
I'm something of an OO zealot and I'm a C# developer so I'm familiar with PS and probably it's target audience so to speak.
PS is a fantastic product but I just find it too verbose. When everything is expressed as an object with properties you need to reference the object and the appropriate property. I know there are alias' for common Bash operations and idiomatic PSallows shortcuts but most people who use PS for short spells to automate something are unlikely to be adopting idiomatic styles.
For PS to gain traction I feel like it needs to be a superset of Bash and offer an easier, intuitive gateway. I know it has enormous potential, I know it has massive capabilities but still I only reach for it as a last resort. I don't think I'm alone in the matter :/
I like Microsoft's move here. They first made available bash on Windows and then PowerShell on Linux. So no one is going to cry. And now they will improve PowerShell and win over users with a good tool rather than just because that's the only one available.
Powershell is an old tech, so browsing the source tree on github should show some of MS internal practices & guidelines in the old times. Yet the github project start in feb 2015.
Where is the source code before that ?
I don't understand Microsoft any more. They are making all their proprietary lock-in stuff open source, and they are alienating their user base. I've heard a number of long-term Microsoft users express frustration the user experience (not to mention the Windows 10 upgrade shenanigans). It was bad enough that my dad, a long-time lover of Windows NT (he liked its VMS legacy) switched to a Mac and loves it.
I feel like Microsoft might be pulling a Sun here. However, I'm sure a lot more likely to use Microsoft stuff if I'm not tied in to Windows.
As a counterpoint, I've been a full-time Linux user (desktop and server) for well over 10 years, with server use for 20. (I'm 40, so while others have used Linux/unix longer than me, this is the entirety of my adult working life.)
With the open-sourced .NET stuff, I've been active in pushing for projects I touch to use it. And, I now dual-boot Windows 10 and Linux, actually using Windows for daily work for the first time in a decade.
Open source C#? Sign me up. And I really do like the Windows 10 interface. The telemetry stuff just doesn't bother me. I think it's largely a red herring.
(And, furthermore, I think privacy stuff really has to get nailed down in laws and society, not just with technology. Going just for a technical solution means you'll always be chasing the answer. Encoding it in laws, and setting people's societal expectations, means you get to set the answer instead of follow behind.)
Finally! It's exactly what we need since there are currently no scripting languages available in Unix user-land for when you need to do something beyond /bin/bash
The initial release is an “alpha” and is community supported. In the future, we will deliver an official Microsoft released version of PowerShell based on open source to anyone running a supported version of Windows or nix. The timing of the official Microsoft version will be based upon community input and business needs. We hope all of you will help us get it right!
'Based on business needs' = if there's ever enough demand.
Windows can have a neat-o shell all they want but when you open powershell on a windows computer you still get the following when you try to ssh... so, not really seeing the point here:
ssh : The term 'ssh' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
he's not wrong though. ssh is the quickest way of accessing remove machines, not to mention secure (i have never seen remote desktop done with priv/pub key in practice). it also makes automation and transferring files easy. putty is terrible compared to ssh on linux. what i would have given for good ssh on windows back when i was still doing C#...
Having lived around the Windows world since Win 3.1, I cannot get over the sneaking suspicion that somewhere deep down Microsoft is working towards an attempt to "embrace and extend" Linux. Maybe it's just me, but they've gotten rather interested in Linux lately and they are reaching out to it in seemly innocuous ways. I hope I am wrong.
I just spent all day trying to figure out why Invoke-SqlCmd only works in certain extremely convoluted ways. I'm going to assume this has more to do with the Sql Server team than it does to do with the powershell team, but still: I cannot be excited about anything to do with Powershell since I'm now up to like 10 Googles per line.
I think PS is miles above CMD but there are still things that bug me. For one, loading scripts into the current session is super confusing to me. Sometimes they are automatically available, other times I have to load them, still others aren't available at all. I'm guessing there is some way to "install" scripts so that they become cmdlets but I don't know what it is.
Another big problem I've had is backgrounding commands. There are multiple ways to do it and I always seem to pick the wrong one. For example, I want to install a bunch of things with Chocolate so I do a lot of Start-Process commands, but they seem to run serially and not in parallel. It's just very difficult to manage jobs in PS, or at least I haven't found any good information on how to do it properly.
A lot of people seem to be missing the point of PowerShell. The concept is far more poweful than the unix shells, because it's a superset of them. It's like Haskell and Haskell-without-types.
We could only dream we had something like strongly-typed pipes in Unix -- and that various tools supported it too.
I dreaded this day would come. As a recent ex-Microsoft alumnus, and with years of System Center insanity behind my back, I saw this coming, and even heard faint echoes of it in the hallways as I left.
I have deep respect for Jeffrey and what he's accomplished (there's a great video of Jeffrey talking to Linux sysadmins and explaining the genesis of PowerShell, including the insane amount of money we burned before he wrote the bits that actually worked over Christmas break), but the grand (corporate) motivation for sponsoring something like this is more driven towards making sure the MS datacenter management tooling (which comprises trendy things like OMS and System Center) attain minimal mindshare in both the on-premises datacenter and on Azure.
That is not something most people would care about, but Enterprise IT likes having a single hammer for all its nails, and PowerShell's promise is to be the one thing that your garden variety IT sysadmin would love to use to automate every single system in your company.
Ironically, PS is effectively lousy at automation because of its inconsistent error handling and incredibly hard to maintain over time due to constant changes and updates, so I'm curious to see how it will evolve to manage all the different abstractions in the UNIX world (knowing my former colleagues, it will likely follow the OMS route of "Windowzifying" those abstractions and making them "uniform" - i.e., Windows-like).
The underlying issue where it regards PowerShell is not the language itself (although yes, something better, more coherent and with an actual standard library would have been nice, like Ruby or Python). It's Windows.
Windows (the enterprise platform) has always been a hectic patchwork of multiple components (AD, SQL, IIS, etc.) that evolve at their own pace, and PowerShell grew to encompass them not merely because basic automation was an actual checkbox we had to have to be able to sign some hefty deals, but, technically because _none of it had open APIs_. Nor would it be feasible at the time to go back and re-do any of it with proper language-agnostic APIs.
It's very nice to iterate over AD entries and set some fields with a couple of lines of PowerShell, yes. But it's also fundamentally impossible to do so with just about anything else.
Tying it all to the .NET stack (which is still prevalent in everything done internally, because that's what customers invested in) turned out to be a lazy-ass way to ensure we could check all the boxes in terms of automating new things as they came along (mostly - WMI is still iffy). Effective and "scriptable", but essentially still proprietary.
And right now, since my former colleagues are betting the farm on Azure, I expect PS for Linux to become the de facto scripting language for managing Linux on it (it already is, to a degree, since PowerShell DSC already lets you "manage" Linux VMs).
For interactive use, PS was a pain to learn (I was a VMS guy, originally), but, most importantly, it was a nightmare to maintain, because cmdlets would pop up and disappear between releases, change semantics, and were quite often inconsistent among themselves.
I shudder at the level of technical debt would-be adopters will be buying into, and wish them luck.
I am pretty excited about this as I have few cross-platform languages that I can use (due to conditions at work). I am wondering how many distros will adopt it into their repos.
cmdlet long parameter names both a) work if you type "the shortest non-clashing prefix", and b) tab complete. And if a parameter takes an enumeration you can tab complete those, too. Which is incredibly nicer than usage: somecmd -umhcglsrfmh line noise and man page lookups.
Actually, you can ctrl+space in latest versions to get a list of parameters (or values if the parameter takes an enumeration) you can scroll around with the arrow keys :)
Is there anything preventing Microsoft from putting a Linux shell natively on Windows? I'm sure PowerShell is awesome, but the DevOps person in me is not excited at all.
Been playing with it ever since I got a new Windows laptop for college (I usually would have installed Debian but I prefer OneNote to notetaking apps on available on Linux). The interesting part is that it's not a VM; the LoW subsystem translates Linux syscalls to Win32 ones
this x100. i wish they'd work on chef/puppet/ansible/salt support instead. probably never going to happen though, so much functionality in windows isn't available from the command line.
You know that the entire point of PowerShell is to make Windows functionality available from the command line, for centralised management and configuration, for automated provisioning on Azure, right?
I think we have different definitions of support. What constantly amazes me is how similar many OSes are - except Windows. DSC hasn't changed that, even though Mr. Snover is a very clever guy. If you're in a homogeneous environment, DSC is probably great. But it's 2016, and environments are heterogenous. Even shipping an SSH client and server with Windows would go a long way as far as interop goes - which may be happening AFAIK (OK, I admit I love SSH, but it's so useful). PowerShell feels a lot like the xkcd n+1 standards thing. I don't use bash any more, but it (along with ash/dash) is the de facto standard.
Sorry I can't get more excited about this, which is especially sad because my background is QBasic -> VB -> C#.
That is how we got started on this. We didn't want to invent anything here. We had a technology called Services for Unix which provided all the shells and utils and I got 99.5% of the way to getting that shipped natively in Windows but it got hung up over IP concerns. So we made it free instead.
It turns out it didn't help managing Windows at all.
The heart of the problem is that Unix is a Document-oriented OS and Windows is an API-oriented OS. In Unix, if you can edit a file and restart a process - you can do most management tasks. Therefore text processing utils like awk, sed, grep are actually management tools. When these became available on Windows - they didn't help manage anything because awk didn't work against the registry, sed didn't work against WMI, grep didn't work against Active Directory. An API-oriented OS needed an API-oriented solution.
That is why we HAD to invent PowerShell. I believed that the Unix automation model was fundamentally correct: Interactive shell composing small tools together in ad hoc ways to quickly construct novel solutions to novel problems. I like to call this the " A | B | C" model.
I did a deep rethink of the "A | B | C" model and asked myself the question WHY? Why not just type A? Why didn't A do what I wanted it to do?
There is the traditional answer about toolchest of tiny tools but I keep chewing on it and came up with a different answer:
A tightly binds 3 steps into 1. It
1) Gets a set of objects
2) Processes those objects
3) Outputs those objects (typically as text)
When when we say A doesn't do what I want it to do, what I'm REALLY saying is that A did one of those 3 steps in a way I didn't want. So piping the results to B and C is REALLY all about taking the text and reverse engineering my way back to the original objects to do one of the steps differently.
My observation was that the pipeline should go between the 3 (smaller) steps and that the pipeline should pass objects and only output to text when you wanted text.
This object orientation is one of the great simplifies of PowerShell. It allows us to shift our focus from the mechanics of text parsing to the semantics of what I want to achieve. We like to call this THINK, TYPE, GET.
THINK about what you want.
Type it
Get it
Imagine you wanted to get all the processes whose workingset was greater than 5mb and sort it by workingset and then format that as a table just showing the name the id and the workingset - in PowerShell, you would do that with this:
So that is the answer - a lot of the great Unix tools don't work on Windows because it is an API-oriented OS. So we had to develop an API-oriented automation model - that is PowerShell.
In the fifth paragraph: "But this is a new Microsoft". They are acknowledging that they made mistakes in the past due to which the missed out big on the server market. As much as .NET is popular, it would have been a lot more popular (vs Java ecosystem) if they were OSS from the beginning. As primarily a Java developer, C# seems more elegant to code in.
As they say - the first step to fixing a problem is admitting that you have a problem.
the Unix shell has been quite static in recent decades; its a sign of success, sharing unstructured text through a pipe is good enough for most administration tasks and people got used to the many command line options of grep/sed/ps and friends.
is this really the best of all possible worlds? I think that its good that the shells get some new competition in the form of PowerShell - maybe some new ideas will come out of it (maybe like passing s-expressions or json around the pipe - something more structured so that we could do with fewer command line options - maybe)
doing structured data 'right' is difficult (right meaning uniform and expressive ways of handling it); historically it was easier to stick with unstructured text - but maybe there is a better way that would be easier to learn and handle.
(Well a major problem of structured data is that there are many possible ways of structuring - leading to more problems for the consumer of the data; maybe it was not adopted in system administration because there is not common/agreed upon culture of how to structure things).
Still need something other than powershell to access a linux server from a windows machine, which seems ironic...
ssh : The term 'ssh' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
Can I write tools that use PowerShell and .NET objects in non-.NET languages, like Python or Perl (using the C implementations, not a .net python)? Like, is there an API that I can implement to interact to implement PowerShell-like "native" tools?
It's not the most efficient way to do it, but if you just want to get it done, Powershell has had e.g. ConvertTo-Json since version 3, I think? You can serialize pretty much anything at that point, and just hand that off to your target language.
If you're talking an FFI though, not so much, I think.
On the Java side here I tend to use groovyConsole when I need to do crazy one off stuff like open excel files which then require a DB query, and maybe to run regexs for little things. I use bash for running docker commands in a specific sequence.
If PowerShell on Linux takes off properly, then that will lead to one of the biggest improvements in the general computing ecosystem in a long time; roughly comparable to the adoption of DVCS, maybe.
PowerShell defaults to UTF-16 encoding, indeed. It's a design choice on a system that uses UTF-16LE as its character encoding and one from the early 2000s. UTF-8 wasn't nearly as prevalent back then. However, the text in both files is still the same.
It's still the same text. Not all of us have to deal with tools that can't handle Unicode or require exact byte-for-byte round trips. Yes, it bugs me too, from time to time, although less so than the extra line break at the end. In that case I have to go through [IO.File]::ReadAllText and ::WriteAllText and thus deal with full paths again. There are a few gotchas.
However, by now we can actually fix that issue. The encoding parameter that the file system provider adds to a bunch of cndlets is an enum and we can add a new value there.
Parsing text is unfortunately not one posh's strenghts... but one of the beauties of posh is that you can very easily include .NET classes, so you could e.g. use the System.IO.StreamReader class https://rkeithhill.wordpress.com/2008/03/02/nothings-perfect...
On the other hand, with posh I could ingest a json file and convert it into an object (or an array of) with ConvertTo-JSON. Once I've got the object, I can query/navigate it instead of parsing it. I'm not sure it is that easy with bash, but then again they are not mutually exclusive, so it's great to have both now on Linux and on Windows.
I wouldn't think of it as a replacement for bash/zsh or other shells. Those are meant to control the OS and run programs where powershell is an environment to programatically interact with API's. They can both fill similar roles but their not the same.
I bought a new laptop recently, and used Windows for a little while (I didn't have time or a USB drive handy for installing Linux for a couple of weeks). I was pleasantly surprised at PowerShell. Not because of all the stuff people normally like about PowerShell; all the PS things are still confusing as hell, to me. But, because MS has embraced compromise and meeting developers where they are in ways that I have never seen from MS until recently.
PowerShell now accepts most of the common UNIX shell commands (this feature could have come at any point in the past decade, or so, as that's how long it has been since I last tried PowerShell, when it was new). The changes that resulted from that change of mindset has made it so I could actually stay in PowerShell long enough to learn other things. Having to re-train my muscle memory away from using stuff like ls, cp, cd, was just a bridge too far, and so in the past I'd revert to cmd (as truly awful as it is) until I could get bash installed from Cygwin.
They seem to have, somewhere along the way (again, it may have been a decade ago, for all I know) given PowerShell a reasonable UNIX shell facade. Sure, it's not quite shell, but bash/POSIX shell isn't flawless; I'm OK with changes, as long as I don't have to think every time I want to do basic stuff that I've been doing for 20+ years with a POSIX or bash shell. PowerShell has some Long-Ass-Command-Names, and I still find them difficult to stomach, but at least all the common sh equivalents (except which, for some reason) already have aliases.
Anyway, the change from a company that tells developers how to work (and for whom the command line takes a back seat to GUI tools and most developers live in the Visual Studio environment, I guess) to one that looks at what's working in all sorts of developer communities and borrows liberally from the best of them, means that having to work on Windows is no longer the worst possible thing I can imagine (I'm even considering porting our products to Windows, which is a suggestiong I have laughed at numerous times over the years).
PowerShell, as it is today, reflects that change, and open sourcing so much stuff lately seems to prove it is sincere rather than merely aping the most visible traits of these OSS communities without actually embracing the single most important facet of them.
I still have an old OSS nerds suspicion of Microsoft, and I will probably never choose Windows as my favorite desktop OS (and I can't even imagine it for server usage...how would you even do that without a decent system package manager?), but my most recent experience with Windows was almost entirely positive. Interoperability has gotten pretty good. SSH is a normal thing, rather than a complicated setup process.
It's interesting, to me, that Apple seems to have been retreating on these fronts. They seem to be less interested in participating in OSS than they were in the early days of Mac OS X. That may be my own ignorance of the platform, but I don't see Apple making these kinds of overtures to Linux/UNIX developers on such a regular basis. Maybe they figure the community is taking care of it for them, because so many devs use Apple laptops even while deploying to Linux servers, so the ecosystem is still pretty strong even without Apple doing much of the work.
"and I can't even imagine it for server usage...how would you even do that without a decent system package manager?"
I'd be pretty hesitant myself to use it as a web-facing server. But if you're running an enterprise network, the reality is almost everything you need on your server can be added and removed from the "Add Roles and Features" wizard. There's not a whole ton of outside packages you would want to add.
If your server is running third party software, you should probably only run that one piece of software on that particular server, and that software should install everything it needs to run on it's own.
It is truly an interactive ("online") environment. Almost everything is typed and through introspection the shell can offer very rich auto-completion hints, without needing tedious scripts such as those used by bash/zsh.
Quick example:
get-<Tab> cycles through all get-[whatever] commands, as you'd expect.
get-certificate <Tab> cycles through all parameters available for get-certificate
get-certificate -ErrorAction <Tab> cycles through the list of valid values for ErrorAction (there's like 6 of them: stop, suspend, etc.)
It has its bad/ugly bits, but the principles are quite solid. In many ways they are what I'd expect if Richie and Thompson would have designed Unix in a modern environment (i.e. not an environment where the output of your commands would be printed :) ).
You can also use things like Get-Command (with options for -Verb, -Noun, -Module filtering) or Get-Member to inspect types/methods/properties. Very helpful when learning a new area.
Scripting. While I typically use Bash on Windows to use grep to interactively test my webserver, I'm much happier using Powershell ISE (which I don't know was ported [1]) to write standard tests with multiple parts, particularly ones that require e.g. checking cookies on the response.
Powershell instead of using grep? Or as a testing framework? Or an embedded interactive REPL?
I mean, just broadly, with basically zero knowledge of powershell, what I see is a custom scripting language with a set of common 'macro' commandlets that do little tasks.
...but, I don't see how that's useful?
How do you import a package to upload a file to S3?
I can't just install https://aws.amazon.com/powershell/ right? ...because that's not ported? So...? etc. etc. for other practical tasks.
Not trolling; just genuinely curious what actual tasks are actually supported that you could actually use this for, beyond trivial stuff like 'check if file exists'?
We are currently performing final validation on our new AWSPowerShell.NetCore module and hope to publish it to the PowerShell Gallery in the next couple of days.
Yes, and there's an extension to use it from VS Code[1]. It works pretty well (autocomplete, F8 to run a script, debugging, go-to-definition, etc.). The fact that it's a language server means it can be used from other editors to (although I don't know if any other extensions exist).
People laugh at me when I tell them I switched "back" to Windows after decades of Unix and Mac programming. But it's a really good, productive platform. Microsoft really seems to be heading in the right direction now.
I hope .NET starts taking off on other platforms, too, because it really is a much better system than Brand "J".
Considering most OSS devs slam analytics on literally every website. Really there isn't much difference.
That's what windows is sending. Analytics information. Yet I can guarantee every major website developed by open source using devs is sending the same information for their apps. Yet no one complains.
Look at the comic. I'm talking about divisions inside Microsoft. I doubt it very much that Scott Hanselman (for example), who is in the camp that pushes Microsoft to open up, is the kind of person that designed the privacy controls (or their lack) in Windows 10.
If you meant Java, say Java please. Brand "J"? To me, that would mean J/K/APL/APL2 (MATLAB, Nial, S, R, and other array systems). But I guess Brand "J" is cute.
I'll join in on the Microsoft bashing club when every other major corporation gets bashed the same way by devs. The day that will happen is when socialism makes more sense to people than capitalism, and it becomes a pervasive idea, but sadly, I think that day won't be coming around any time soon.
I've been enjoying the fruits of Bell Labs and UC Berkeley.
I'm not a "dev" or even a Linux or Bash user. I like the simplest Bourne shells the best. Plan 9's rc is also good.
I've never enjoyed using Microsoft Windows. (But if they gave away the source for NT kernel I might find use for it, after a lot of work trimming it down to size.)
Maybe the size and power of UNIX has spoiled me.
MS obsessively focuses on their competition to the detriment of users and software quality.
He more or less said that the GPL is cancer which is pretty accurate. The entire quote is
"Linux is not in the public domain. Linux is a cancer that attaches itself in an intellectual property sense to everything it touches. That's the way that the license works"
We detached this subthread from https://news.ycombinator.com/item?id=12313389 and marked it off-topic. Baseless accusations of astroturfing are not allowed on HN. If you believe this is happening and you have evidence, please email us hn@ycombinator.com and we will happily investigate.
Powershell is a horrible abortion, what the hell do people see in it? I mean that on a language level, I cant speak for Windows specific features.
I have once written a simple script to copy a folder recursively to a fileshare, rename some stuff, edit some files. Took me 4-5 hours.
I then did it with julia in 30 minutes and I never did anything in either language before that.
It's insane the kind of weird language crap you have to deal with, the weird options that some functions need and worst of all: That the functions have weird problems, like I dont remember the specifics but I'm pretty sure that the folder copy function had no option to actually recursively copy all of them. I had to work around that. Then I had to work around some more specifics around file/folders.
In a normal language you would just call something like "copyFolder(srcPath, dstPath, recursive: true)" nope.
I wouldn't go so far as to call it a "horrible abortion" - but it can be painful to use at times. I manage several Azure environments with PowerShell and not a day goes by that I don't wish there was something better. Especially on those days when I have to dig into Azure's JSON templates.
There is an SDK for Python, but I haven't had an opportunity to try it. If anyone has used it I would love to hear about your experience.
Linux ecosystem has a huge amount of different tools and scripts for most of the situation that you can imagine. I'm working with Linux more than 14 years, so I can say that text processing idea is the best way. It's pure KISS principle. If you need just to see something - type a command, for complex operations - awk/perl/python. PowerShell looks ugly because it's something between a programming language and shell. It's not KISS. It's all in one tool.
There are the Azure REST APIs that you can curl. Or the Azure-CLI on Linux and Windows.
That said, I've noticed that powershell seems to get priority with Azure support before the azure-Cli client, so there are a few gaps here and there. I've been told that the Azure REST APIs are all in step with Powershell support, but sometimes go in documented unfortunately.
To be honest, I prefer PS to the Azure CLI but I have had to resort to the CLI on a couple of occasions. If I remember correctly it was to reset credentials on a VM (which wasn't possible with PS at the time). I guess the main problem with the CLI is that it's not a shell or a general purpose programming language.
The Azure REST API requires a ton of deeply nested JSON data structures. Take a look at this [1] sample. I just can't bring myself to work with that every day.
I really need to buckle down and try that python API.
Azure CLI, Azure PowerShell and even the portal are nothing more than API clients. You can capture traffic while working with any of them and see it for yourself.
This might come across as a little rude, but I really wish Windows developers stayed away from Unix. I moved away from Windows a long time ago because I didn't want anything to do with Windows or Microsoft, and I went to great lengths to avoid their software and their proprietary file formats. Now we had command-line interfaces, desktop environments, running on top of an incredible, powerful Unix-like OS. It was free software, and it was Microsoft-free. Please, don't ruin it.
"Um excuse me, but I really like dogs. However, I don't like you, so I don't want you making movies about dogs. Dogs are my thing. Stay away from the concept of dogs."
GitHub is just the distribution here, and software licenses are subject to change. That's the embrace portion.
Hypothetically, Powershell becomes widely used on Unix, and Microsoft releases Powershell 2, which extends Powershell. This version only runs on Windows and contains killer features. These new features can't be implemented from the GitHub source either technically or because of intellectual property. That's the extinguish step.
The purpose of EEE is to extinguish a technology. Your situation doesn't hold up because PowerShell is MIT licensed. Even is Microsoft did release a new version that was Windows only (which, given their use case, makes no sense), there's nothing stopping the community from forking the repo and adding those features, or porting those features to another technology.
People will continue to trumpet the sound EEE banner every time Microsoft does something that people like. The problem is that waging an EEE campaign with MIT licensed technologies on Github is extremely ineffective. Also, it's pretty clear that while Windows is still a big part of Microsoft, they are trying to move beyond the desktop. The rise of macOS, the proliferation of mobile devices with iOS and Android, all these developments have caused the company to shift its strategy to the cloud, namely Azure. Even in that space, they can't wage an EEE campaign, because with competitors like Amazon and Google, there's no way they can lock you into Azure
Times have changed, and so has the company's strategy. Whether or not the company's culture has changed is not something I am concerned with. They have enough watchers that call them out, like with the Windows 10 update debacle and the collection of telemetry data. I am more interested in their actions with regards to their technology, because it's my opinion that have some of the best people working on languages and tools.
Cool. https://powershell.org/ needs some work though. It screams WordPress 2008 website, and I'm not given any idea how to install or use it, or what it's good for, just by looking at the front page.
This could really be used as the default reply for a lot comments here that don't get why people are still a bit antsy about MS apparently doing things for the good of others besides MS. But it's probably an argument only worth having in one thread. I can't really add much to the argument, other than I think my feeling matches a lot of others in the OSS world: ever since Nadella took the helm I've been cautiously optimistic that we're not just part of a very long embrace+extend cycle that will end with a crushing extinguish, additionally as the years go by without an attempt at extinguishing but even more embracing+extending instead, with MS having lost so much of its former power and weight in the industry as a whole, I think even an extinguish attempt wouldn't be all that successful. It's not enough for me to actually use these offerings, but I'm not going to go on a rant that no one should use them or distros should avoid packaging them. (e.g. Stallman once had a Mono+Banshee rant that was probably justified at the time but it's hard to argue the position now.)
Come to the dark side, we have (strongly-typed) cookies.