+1 for fish, the autocompletion alone makes it my daily driver. You can get the same in any reputable shell, but I like that it is designed into it from the ground up (the zsh autocompletion plugin brands itself as "fish-like").
It's also compatible-enough with POSIX for most purposes, and supported as an alternative for lots of software (venv comes to mind).
One area where fish is excellent is IMO interactive shell scripting. No more case..esac and friends. Fish adopts a python-like syntax that makes it ridiculously easy to write loops fast in the terminal.
So I read this and thought "wow, indentation would be so stupid to manage in a terminal... even typing python into python's repl is brutal" but then I looked up the syntax for a for loop in fish and it is exactly the same as the syntax in bash except it uses the word "end" instead of "done" and doesn't have "do"... what am I missing?
It's an absolute godsend. Scripting in it feels really intuitive compared to POSIX shells. I find it such a shame that we probably won't get rid of POSIX shells for at least a couple more decades... if that.
How often do you find yourself forced to use bash? ssh/machine init scripts/etc. I could never use an alternative shell full time because I need to stay good at bash.
I've been using fish at home and bash at work for almost a decade now, and it's been fine — and because I'm at a fish shell on my own computer far more often than I'm at a bash shell on a work server, I'm willing to take the slight productivity hit there, because I gain overall.
The thing is, when I picked up fish, I couldn't forget my existing bash knowledge. I still have to write normal bash-dialect shell scripts, and with the fish syntax being so much nicer and simpler, the two can happily co-exist in my head. Occasionally I'll hit Up to search through history instead of Control-R, but I quickly realise what I've done and use the bash keybinding instead. And the only fish aliases I missed were 'l', 'll', and 'la', which I just manually copied over without bothering with the rest.
If I had to use bash more often at work... I'd still use fish at home after all the time I've sunk into it, but YMMV.
It depends on your role. Many people regularly maintain or interact with prod deployments which cannot be customized to your liking. So it may be "somewhere" or occasional in your case but not in all cases. Switching back and forth between shell syntax every few hours is very disorienting and worse, can cause serious errors (for ex: if I have a local alias with rm="rm -i" but a prod server doesn't have it)
POSIX compatibility would be enough to make oil a legitimate replacement shell for me but the bash compatibility makes it stand out as an alternative to take seriously.
Does oil allow a gradual upgrade path to using oil semantics for stuff like “echo $dontbreak”? Are there “set” options that enable oil semantics? Sort of like “use strict” in JavaScript and perl
>I could never use an alternative shell full time because I need to stay good at bash.
Getting "good at bash" is a game with diminishing returns.
It's so random and non-orthogonal (bash itself, and the flags of common UNIX cli utils) as a scripting language, that you always end up having to look this or that, and with crazy corner cases you've missed.
The most common tripping points are the minor adjustments in syntax, probably the biggest one being command substitution:
Bash:
$ echo “$(ls -al)”
Fish:
$ echo (ls -al)
You get used to it though.
Another thing many bash-ers sometimes miss is the !! substitution, e.g.
$ apt update
# permission denied
$ sudo !!
This doesn’t work in fish, BUT fish does offer a handy keybinding out of the box to prepend ‘sudo’ to your current command line (submitted by yours truly). So when using fish, you’d handle the above scenario like this:
$ apt update
# permission denied
# press Up or Ctrl-P followed by Alt-s
$ sudo apt update
There are so many other great things I love about fish, but this comment is already long enough.
Be sure to read (or at least scan) through the user guide on their website. It’s an easy read and not dense. You can also view it at any time by running `help` from inside fish.
I can’t recommend fish highly enough, I’m excited for your journey :)
Adding this to your config.fish will give you !! and !$ in fish
function bind_bang
switch (commandline -t)[-1]
case "!"
commandline -t $history[1]; commandline -f repaint
case "*"
commandline -i !
end
end
function bind_dollar
switch (commandline -t)[-1]
case "!"
commandline -t ""
commandline -f history-token-search-backward
case "*"
commandline -i '$'
end
end
function fish_user_key_bindings
bind ! bind_bang
bind '$' bind_dollar
end
I’ve been a fish user forever, but the lack of support for sudo !! has never made sense to me. I’m glad to finally know that there’s a 2-keystroke alternative.
In regular Bash just press up and opt-up. If that doesn’t work your key bindings are bad and you should fix them. I never understood why anyone would ever use !! instead of something that is fewer key presses and more universal.
I used fish for two years or so, and recently switched back to zsh, mainly because Fish is too slow to load, and this is not something the developers are willing to fix. The problem I had was that I needed to load a “bare” shell, without plugins or anything else, but this is not functionality fish provides, and judging from the developers’ comments something they are not going to.
So I spent a day migrating my fish config to zsh, and am much happier now.
This is interesting as I had the exact opposite experience. Fish is screaming fast for me compared to zsh, primarily because I don’t need to use any plugins to achieve all of the functionality that I needed to use many plugins to achieve in zsh. The totality of my fish config files, not including functions, is maybe a couple dozen lines [1].
Regarding a bare shell: perhaps I’m not understanding your exact needs, but fish is able to check whether or not it’s being used interactively with `status is-interactive`, so you could certainly try wrapping all of the plug-in loading machinery around a guard like this.
You might want bass (https://github.com/edc/bass) or replay.fish (https://github.com/jorgebucaran/replay.fish) if you need to run bash scripts that modify the shell. I personally use replay.fish (although I guess I haven't updated in a while since my version is still called "bax").
In fish, tab does normal completion. Right-arrow accepts the current suggestion. You can use Ctrl-F instead of right-arrow (as usual for Emacs-inspired movement keys).
Edited to add: A suggestion is a complete command line based on the history. Completion looks at the disk for file names and the like, and only talks about the current "word".
If the '→' key is too far away from the home row for your liking, you can use Control-E instead, as long as you're at the end of the input buffer (it's the "go-to-end-of-buffer" command normally)
Rash is a shell embedded in Racket that was presented in a paper at GPCE18[1]. Oil is cited in the paper, so Andy really should include it in the list as well ;-). There are also a few others mentioned in the "Risky review of related work" section of the paper that aren't on Andy's list yet, see the paper.
Another, similar and minimalist "shell language" is janet-sh[2].
Also, this suggests a slightly different form of categorization for Andy's page, i.e. new shell languages vs. "embdedded" shell languages which are libraries that add just enough functionality to a parent language that it becomes useful as both an interactive shell and a scripting language. Xonsh would go into this category then instead of the somewhat poorly defined "interactive shells" category, since Python with Xonsh is a better shell scripting language than Python by itself.
Thanks for the list! For completeness, I think the active older POSIX shells include at least NetBSD's sh, dash, and bash. Not sure if there are any others.
Tried an alternate shell for a while, ran into the basic problem that the moment I SSH'd somewhere else it was gone, and my instincts for how to get things done expediently were degrading.
If I was going to use an alternate shell, it's got to have some solution for this problem that's transparent - something I can load into any sh/bash environment to get it running remotely.
This is the core problem with so many software efforts now, there are just too many efforts that portability suffers.
I remember years ago mucking around with my home computing environment, making it so convenient, custom UI shell, monitoring widgets, highly customized bash environment with all sorts of fancy utilities that were just so handy, and all sorts of nifty features everywhere to tackle any problem I encountered.
Alas, the second I went anywhere else to do something--some friend or business's system, none of that underlying infrastructure existed. I lamented, "oh if only these other people followed my model I could just use X right now," life would be better.
At some point in my career I realized I was just wasting a whole lot of time customizing my native environment because those features were non-portable and the more I dealt with working in other people's environment, the more I realized I should just focus on learning the best way to do things in the popularly adopted environments because it meant my knowledge transferred almost everywhere.
I'm the same way, and I don't think it's ideal. I tend to do my work with the minimal set of tools that I usually expect to be available, and I'm extremely biased towards using the default settings and configurations of those tools. I do have settings that I really prefer so I have to use, but I naturally keep them to a short list that I can set up within a minute or five. I can tear them down in a minute or five also - I may not be using my account; I often find myself on Company X's account installing Company Y's software on Company Z's servers, and I wouldn't want someone else coming after me to be confused.
I don't even really change the wallpaper on a desktop anymore, feels like polishing a hammer, a waste of time.
I wish you could carry your environment around with you, but I don't see how it could possibly work that way neatly.
An exception is that I usually end up installing a lot of stuff on Windows desktops to make them vaguely usable and scriptable, but Powershell and WSL will probably eventually mean I give up doing that, too. The previous sentiment was really about UNIXy systems.
It won't immediately solve the problem, but it can eventually. It can be the default and only shell on a Linux distro, whereas shells like fish can't, because they would make the system non-POSIX.
This is why I ended up sticking with zsh after I tried fish for a period. I liked some of fish's features, so I used the two plugins fast-syntax-highlighting and zsh-autosuggestions to get the same features in zsh.
It'd be cool if there was something like VSCode's model where it set up some sort of daemon that provided a shell environment (instead of an editor) remotely. Perhaps mosh can do something like that?
Or even just a script which could quickly bootstrap a shell and dotfiles on a system.
I always wished for a ssh environment that worked more like a client server app. In other words you could connect to the remote server but your shell is still the one that take the calls and dispatches them like api requests. The remote server can still be as secure as the api exposed is only the ones that the user has permission for. The big difference is that your shell and tools are always the same and it would allow to easily run commands both locally and remotely in the same shell/session. More easily share data. There are probably good reasons to not do this but I think it would be a far better user experience.
something like Plan9's cpu server would help here: you "mount" your shell/IDE environment from your computer and "mount" the other computer's cpu, memory, and fs over ssh. "mount" is in quotation because in Plan9 you not only can mount fs but cpu, network interfaces, desktop windows and what not too.
i think they are all wrong - a next gen shell needs to focus on structured data - hacks like jq and yq need to be made redundant as being part of the shell. I didn't manage to pursuade redhat that this is going to be a priority (i used to work for them); i still think that this is going to be a priority in the near future.
There are a couple which attempt this. I think nushell works that way. And powershell, though it sacrifices language independence by using .net objects as its structure.
I've been working in a Windows shop for 2 years and I'm to the point where I just tell people to rewrite things in Powershell. It has native json support so yes, jq is obsolete.
On this list, I've looked at elvish before, it seems like it also is designed to make jq obsolete, and it is much more posixy than Powershell (though I think parts of posix really hold shells like Oil back from being as powerful as Powershell is and something like elvish may be.)
On that page you write that “you _can_ use oil in production but ...” which sounds like you recommend against it.
Later you write that oil is best for automation and fish is better for interactive usage.
This got me confused. Do you recommend that I run fish in my terminal, rewrite my one liners to oil when they outgrow their one line, but use bash in production?
With that being said I really appreciate how honestly you write about the limitations of oil and benefits of competing shells. Few project seems to be as open minded about their competitors and aware of their own short comings as oil, which in my opinion is what you want out of something you put your trust in.
Yeah I would like Oil to be the best of both worlds, but realistically it won't be in 2021. That dichotomy is sort of explained at the top of the FAQ: shell as a language vs. shell as a UI.
(I plan to use bash interactively and Oil for scripting for awhile, which is natural because Oil runs bash scripts. Others may want to use fish; fish is too incompatible for me.)
However in 2021 I hope to remove the qualifier about slowness. That's the only reason I don't use it in production now! (There are several Oil-only features that I want to use in production, like better tracing.)
---
I've written several times on the blog about how the scope of the project is too big, especially in the last year :) So the goal is really to have Oil be the best for ONE use case, and I think that's "cloud automation". I will run Oil's own extensive continuous build to Oil as part of this.
And once people are using it for that, it should attract contributors, which will address the "scope" problem.
Oil is very compatible with bash -- the most compatible by a mile. It's very modular, and I mentioned in this thread how you could do things with it you can't do with bash.
So basically everything about the project is working, but it takes a long time. The thesis from the beginning has panned out -- otherwise I would have stopped working on it already :)
It turns out it really is the best thing for what I want to do, and I'm confident that will generalize. Shell is becoming more important, not less important! (I have some upcoming posts about this, i.e. regarding distributed systems and "the YAML problem".) We need a better shell -- that's a smooth upgrade from POSIX shell and bash.
I'm working on creating a modern terminal interface with Electron where you can use the mouse to modify the command line, each command sends output to a separate "output block" (think Jupyter notebook), you have global autocompletion from history regardless of which machine you are sshed into or how many shells deep you are, and where every time you start typing a command the man page automatically appears in a sidebar.
This is surprisingly hard to implement. If you just run bash unmodified as the backend then you can't multiplex the output from different commands and therefore you can't display output from different commands in different blocks. If you run each command in a separate shell then you lose automatic tracking of environment variables and working directories... (There are other challenges but they are a little easier to work around.)
Does anyone have creative ideas for how to work around this?
(Oil author here) Yeah I think you will run into many problems doing this in bash, and other people have.
A few people are working on similar UIs for shell, which I think is a great idea. I would like you all to get together and tell me what kind of API or IPC Oil needs to support it. And maybe hack on the code too :)
There is some nacsent / working code out there, linked in this issue.
I've been working on my own terminal for a while now, and capturing the output of separate commands and reusing it is one it its unique features. See Extraterm in [action here](http://extraterm.org/features.html#shell-integration)
It doesn't quite do what you are asking though. There is shell integration for some fancy features and it works with the most common 'normal' shells. I don't have a custom shell or similar which deeply integrates with Extraterm to automatically run commands in separate panes or windows.
Extraterm is based around the idea of a stack of 'blocks' which can hold output, images or more complex data (possiblye tabular data) etc. Although I haven't tried it directly, emulating something like a Jupyter notebook should be possible after writing some extensions via Extraterm's extension API.
A session spanning 'global' history would be fairly easy to implement as an Extraterm extension.
Getting what you in particular are asking for to work across ssh, containers, etc is difficult. You need to run your own code (=shell or "command runner") on the remote end somehow and you don't want a manual installation step everywhere. The only thing left is doing some kind of over the terminal delivery of code to the remote shell and then running it.
I think something like this would be good, but I don’t think I’d base it on bash. Some of the things I would have are:
1. When someone writes a command like foo | bar, instead of making a single pipe from one to the next, put your own lightweight program that works a bit like pv in the middle. In the standard case this is a pretty cheap program: it just needs to call splice to move from stdin to stdout, but you could have some mechanism to eg view the data going through a pipe or the rate of data: one could tell if the program is doing anything
2. Typical interactive shell use involves writing commands, inspecting output, then modifying the commands. A notebook format may be better for that. I think I’d want some mechanism to say “these commands will generate some output. I want to inspect it then write the next stage in the pipeline so hang onto that output instead of rerunning the command.” And why not make some “interactive grep” and interactive versions of other commands so that one can get immediate feedback when modifying the regex.
3. It’s hard to know if a command will have side effects but I feel like it matters a lot. Maybe something could be done to detect it somehow.
Hi. I'm Ilya, author of NGS.I see there is (at least) some overlap between the UI you are thinking about and the one that I'm thinking about - https://github.com/ngs-lang/ngs/wiki/UI-Design
The plan for NGS is to support multiple UIs. I was thinking terminal and web.
I would also (as the author of Oil shell) be glad to hear about APIs and protocols.
> start typing a command the man page automatically appears in a sidebar.
How about this one (unknown amount of work but I was thinking about this in NGS) - parse the man page in order to translate an exit code to a human-readable error message?
In addition to the highlighting of input vs prompt, note the little triangles which allow you to hide parts of the output. Try it - it's live. The triangles at the right of the screen are emitted by bash (see instructions here https://domterm.org/Shell-prompts.html) or the Kawa Scheme REPL. The triangles elsewhere are emitted by the application (PrintNested or Kawa) to display a nested data structure. Note the latter does dynamic pretty-printing - re-size the window and the line-breaks are recalculated by JavaScript in a way that shows the logical structure of the output. (This is done by the application sending some special escape sequences with the output. The escape sequences are recorded on the DomTerm data structure, and handle re-flow for commands that are dead.)
Also notice the support for nested command groups: Clicking the button by the "kawa" (shell) command hides all the non-initial lines from the Kawa command, while buttons by individual Kawa input lines just hide that Kawa command.
What I'd like to add is a hook for the "job" feature of various shells. When you put a job into the background, it would hide the output from the command, which would continue to produce hidden output. The shell would emit a new prompt for a new command. All background jobs could be made visible at the same the (if you want) by clicking the show/hide triangles, and hidden by re-clicking.
One way a shell could do this is by creating a new pty when creating a new job. DomTerm could be set up to interleave the output front different ptys. (DomTerm could build on its "buffer" model, which extends the xterm main/alternate buffer model. DomTerm supports an arbitrary number of buffers, which are displayed one below the other. Interleaving pty could build on this by tying each pty to a new buffer.)
I have been thinking about an alternative to bash/POSIX style shells, and I find powershell kind of genius in this regard.
What could potentially work for the linux world is perhaps dbus objects between piped programs (they are easy to read/write even from lower level languages) and some way for programs to communicate whether they are capable of interpreting objects, or should fall back to string based communication. And at the boundaries there could be automatic translations (though object to string is easier) with something like toString.
So for example `ls -la | filter f -> f.date > 2021-01-01 | grep something` would either shadow ls to a new shell-aware implementation or patch it /interpret automagically the table layout and then the resulting list of objects could be filtered and at the end reconverted to an object “toString” per each line which can be operated on by grep
PowerShell runs on Linux. I run PS as my primary shell [0] on Linux today. It's is open source too.
You're not far away with how PowerShell can parse Linux commands. Instead of relying on awk/grep, PowerShell uses .NET under the hood to provide consistency across operating systems. There are a few methods for converting from a Linux command to PowerShell objects based on delimiters or OFS. If you're interested, Steve Lee wrote a blog post that does a good job covering how to parse Linux command output into PowerShell objects [1].
M 0.12.7 jao@cheese ~$ import datetime datetime
M 0.12.7 jao@cheese ~$ ls | select (f: f.mtime > datetime(2021, 1, 1).timestamp()) | grep py
.marcel.py
.python_history
ls is builtin (and has -la behavior by default), select is provided by marcel, and grep is from the host OS. Note that the python function datetime.datetime is imported so that it can be used in the select predicate.
POSIX commands are available on *nix platforms and PowerShell provides native functions or POSIX wrappers for most tasks. About the biggest issue is speed, but there's always POSIX or .NET Core in the rare instance tons of performance is required.
If I may, PS is a first-class citizen on Linux, macOS, and Windows. PowerShell is a spectacular cross-platform shell that provides an object oriented paradigm and it's available today.
A bit pedantic, but I think this statement "Probably the most popular non-POSIX shell" about fish may need this qualifier "For Linux Systems ...."
tcsh is the default in FreeBSD, and csh/ksh is very popular on proprietary UNIX systems. Practically no one proprietary systems move outside of the default (I see this a lot in AIX, which defaults to ksh).
I’ve been trying many of those but failing to switch because I’m too brainwashed by bash (even zsh trips me up when some non-bashisms happen to trigger) and ended up writing one toy shell for me to peruse.
Goals/Plans/Ideas/Experiments:
- interactive, compatible with the subset of bash I use the most
- written in Ruby, forks only when necessary (e.g for subshells uses Ruby threads and pipes to spawn and connect new Shell instances)
- extensible via Ruby (i.e can define new functions in Ruby, accessible more or less directly from the shell)
- fzf support
- can be built against mruby to embed a Ruby VM + code in a single binary and not require any Ruby installed
- can be assembled into a single .rb to easily curl it into any machine where there’s a Ruby already
- no dependencies
- some sort of concurrent/async evaluation for fast start, prompt, autocompletion...
Still a long way to go, but I find the experience rewarding even if it ends up going nowhere.
Not to be confused with Windows Alternative shells. I always enjoy dropping these links for folks who didn't know previously that this was a thing at a point in time, and that at one point the definition of "shell" between Windows and *nix was a bit different:
chubot, I have to say I really admire your dedication! I only follow oilshell to the extent that I read your articles and dip in to the wiki, but even that is a joyful learning experience.
rc shell is still maintained for Plan9 (in both 9legacy and 9front) and for UNIX in plan9port. I have used it has my daily shell for almost a decade now. There are also native UNIX ports available, usually based on https://github.com/rakitzis/rc.
I enjoy the simple syntax and I wouldn't call it dormant by any means. It kept simple on purpose and doesn't require many changes.
Ch is actually proprietary software, if it was free software, I would probably use it as my login shell. I spent some time writing scripts and customizing my .chrc and really enjoyed it. It's actually a C interpreter, but with shell features.
Okay, can anyone point out some real day-to-day use case for which I should ditch fish and move to something else? I need all the niceties of fish and something more.
I used elvish for as my daily shell for a while, and loved a lot about it! Eventually made the switch to fish, which is also wonderful.. can't remember exactly why I stopped using elvish. Ultimately ended up moving to zsh because posix compliance is convenient.
Genuine question: I thought that operating systems only have one shell, but that can be accessed by multiple terminals? So this should be alternative terminals?
I'm not trying to be a pedant I just thought I understood something, now Im not so sure.
In the case of Linux (and other Unixish OSes), getty is the program that makes a terminal or console session work on the host side. Getty usually launches a program, login, that takes your username and password, retrieves your account's settings and launches your interactive shell (something like bash, zsh, fish or oil) when you successfully authenticate. Incidentally, tools like screen and byobu allow multiple terminals to attach to the same shell by multiplexing the terminal (not the shell).
A unix/linux OS can have multiple shells. The shell you land in at login is determined by the shell listed in your account's /etc/passwd entry and can often be changed via chsh command.
I was thinking of extending my shell (desh) so that stores the environment and history on a server so that a shell started on any host in a network could optionally stay in sync with the other instances. Eg. ENV={ssh host envstore} ; HIST={ssh host histstore}
“shell” is kind of a loosely defined term in this context, but i just think of it as a textual interface to the whole system. really it’s just a program that accepts text commands (or scripts) and performs actions on the system or launches other programs, and manipulates and displays the results. eg: sh bash zsh fish xonsh
“terminal” is pretty well defined as a graphical UI program which presents you with the shell of your choosing (historically related to teletype or whatever) eg: xterm Terminal.app Konsole iTerm alacritty
for the most part you can use any terminal with any shell, so the terms kind of coalesce into the more generic idea of a “command line”
also all of this is for unix-like systems, eg linux macos bsd. windows is... well windows
I am really confused about oilshell. Is it ready for me to install and become my daily driver? What does it do better? Is the syntax different? It would be really nice to see code snippets on the front page like fish shell's website.
Even the "How is Oil different than bash or zsh?" section doesn't really tell me anything about it:
> Oil is taking shell seriously as a programming language, rather than treating it as a text-based UI that can be abused to write programs.
* NGS is programming language first, as opposed to most (or all?) alternatives. A language designed specifically for Ops. In practice it means geared towards particular use cases - https://github.com/ngs-lang/ngs/wiki/Use-Cases .
* Your day-to-day pain factored out: how many times you are supposed to define warn(), die(), log(), retry() and what not?
* The language is in OK shape, we use it at work.
* The work on the shell (CLI) is just starting. The shell will be in NGS language itself.
It's also compatible-enough with POSIX for most purposes, and supported as an alternative for lots of software (venv comes to mind).