Digging around in terminfo is like holding a flickering, flaming torch up to some hieroglyphics in some deep catacomb. It's weird to think just how much complexity goes into a terminal application. Makes me even more grateful to the devs behind iTerm and Kitty (and all the others!)
EDIT2: Contrast terminfo with the old-school method of connecting to HTTP or SMTP with telnet. Yeah, it's simple text but that text is being displayed by a complex application that supports all sorts of wild stuff!
I use my own terminal, and early made the decision not to give a shit about terminfo, pretend to be rxvt, and deal with the fallout (now, admittedly, the only user I care about for my terminal is me - at least for now). Modern terminals supports enough of a shared subset that it's less painful doing that than having to deal with applications get all confused because they don't recognize your TERM variable, to the point where I've typically found the closest match whenever I've used a less common terminal in the past too. (This matters less if you don't regularly ssh into new systems, of course)
Absolutely agreed. The main problem with that is providing new features that xterm doesn't support, but terminfo pretty much refuses to add new features anyway, so modern terminals have had to work around that with some combination of autodetection, separate environment variables, or YOLO approaches. So, at this point, I'm not sure it's worth supporting terminfo at all for most applications, either as a terminal emulator or as a CLI program. (The article makes a similar point about the negative value terminfo is providing for modern applications.)
If you're building a program that takes over the terminal screen, you probably should use one of the established libraries that abstracts over terminfo and over old non-ANSI Windows consoles. But if you're doing light terminal handling in an application that doesn't take over the screen, such as emitting colors or doing simple cursor control, forget about terminfo and just handle two or three cases: ANSI, optionally old pre-ANSI Windows consoles, and files/terminals with no support for anything. Rounding ancient terminals or terminal emulators to "no support" and giving them the same thing you'd give a redirection to a file seems entirely reasonable for a new program.
I'd frankly far prefer people not to call isatty(), and require switches for machine-readable output. The kind of tools that "barf ANSI cruft" are usually the tools I want to do so even when I'm piping their output into something else. It's a real nuisance to e.g. deal with options to preserve colours through a pipe for every element. There might well be exceptions where I'd prefer to default to checking isatty(), but they are few.
If there was any remotely standard way of signaling the preferred default, and a remotely standard command line option to toggle that default, it'd make things a lot nicer, because I totally understand where you're coming from; I get equally annoyed just in the opposite scenarios, so the situation is a nuisance for both of us.
I'd love to have a standard command-line tool for "run the following program with its output set to a pty, and then pipe the output to stdout". That's the most reliable way to get programs to display color and similar.
Any tool whose output is ever parsed, including "parsing" as simple as "tool | grep xyz", shouldn't emit terminal escapes to a pipe by default. If `xyz` has an embedded color sequence in it, that grep will fail. Or worse, produce unexpected results. (The standard color sequences end in `m`; a grep for 'msomething' could match 'something' preceded by a color sequence.)
I agree, of sorts. The problem being that we have developed a culture of making user-facing user interface and APIs the same in Unix-y tools. I think "everyone" recognises the problem, but we may have different preferences about what the default should be. Personally, I'd prefer a default way of indicating "I want API behaviour now" but defaulting to more human-friendly output, because even in most cases like "tool | grep xyz" from the command line I'm still more likely to want colours etc. to stay intact through the pipeline. Most of the time that's fine, but you're of course right it will give unexpected results some time.
I think my ideal expected behaviour (which would still not be perfect) would be something like:
* Tools defaulting to unescaped output in scripts, but with a standard short option and/or ENV var to trigger colour output.
* Tools default to colour/escaped output when run in an interactive shell even if in a pipe.
* Tools being escape sequence aware, maybe with switches to turn that behaviour off if you genuinely e.g. do want to grep for sequences that may include escapes and you want them considered.
But I'm not sure there is a good solution to this other than decoupling UI and API and having different defaults for tools that are expected to be "user facing" vs treated as API. I have an "ls" replacement on my system, for example, which changes formatting and adds more colour to my ls output, and it's obviously not named ls because the amount of stuff that breaks if "ls" isn't reliably the same as always is significant. It's still not great to have to separate this given that part of the ease of composing pipelines etc. is familiarity with the output, but maybe if coupling that with reasonably standard switches to turn on/off machine-friendly output.
I can imagine that a world like that could work (with some care to make it less error-prone for common command-line scripting, since command lines can sometimes be small scripts and evolve into large scripts). But I think it's too late to do that for CLI tools without causing widespread breakage.
I think we might be able to do better in something that isn't a traditional shell, and that uses ptys instead of pipes, together with builtins that replace standard UNIX tools with escape-aware tools.
Yeah, I think it at least means abandoning any idea of changing the API of any of the standard tools. Whether you silently replace them in a custom shell or provide alternative named replacement.
I'm halfway tempted to replace my shell with one that is more integrated with my terminal and do something like the last bit you suggested, given it can be very trivial[1] if you explicitly make the choice that for any scripting you'll use a "regular" shell.
[1] there is, in fact, a tiny single-file Ruby shell that I might be tempted to extend.
You could write a wrapper around something like https://github.com/cvolny/faketty, which can manipulate isatty() calls when used with LD_PRELOAD on Linux (and presumably DYLD_INSERT_LIBRARIES on macOS except on Apple's binaries).
Also, as a pet peeve: always read your input from stdin, and if stdin isn't readable, do not assume that if stdout/stderr is a tty you're allowed to use that for input. (This assumption is broken in batch/CI/etc systems where stdout/stderr may be a tty so that a program emits color/etc, while making stdin /dev/null because there's no user interaction possible.)
TTY queries are written to stdout but read from stdin. That's not user interaction. E.g. if you're system doesn't have ioctl for window size (or you're over a remote serial etc), setting the cursor to the bottom right and asking it's position. Those programs break with no stdin because a tty is inherently bidirectional communication!
Such queries are not universally supported, and a program using them has to be prepared to expect a lack of response to the query. (This is one of many issues with such queries.)
Frankly, I find most work well enough with xterm or rxvt termcaps to prefer that over having apps just refuse to even try to work until you've changed settings.
But conversely, if distributed like that, I'd also feel that this implicitly means any failure to act like xterm is a bug they've implicitly suggested it is reasonable to report or complain about (and if the version number doesn't imply it's an early stage release, and it still doesn't work well with its defaults, I'd get cranky)
If you send feature-detection queries it doesn't matter.
The nasty part is that RXVT violates ISO 2022 structure in weird ways. It's not alone in that, but most of the other-program violations are much more easily fixed.
When I was on ITS, I just implemented SUPDUP in my terminal. It made Emacs usable at 300 baud. That protocol is a nice piece of engineering.
I got a big dose of Telecoms standards early in my career, including some X.25 and networking stuff. After the clarity and simplicity of SUPDUP, my reaction to Big Standards could best be described as "allergic".
[Marshall Rose and Michael Padlipski are good reading on this subject. To this day I use the word "octet" as an epithet]
That's fascinating! Did you have a specific itch you wanted to scratch by writing your own terminal? Or was it a learning project that you found useful?
A combination of a procrastination project, wanting a pure Ruby library I can use for output from hybrid applications (e.g. my editor) to be able to have a mostly terminal UI but being able to add in limited graphics when available, and wanting a testbed for other terminal changes I want to experiment with in terms of creating more of a hybrid UI.
A basic terminal can be fairly small - mine is ~1800 lines of Ruby at the moment, which I consider disappointingly large. For comparison st is ~8k lines, I think, and xterm is ~88k. But you can do a working terminal in far less than my 1800, even in more verbose languages, so it's a nice space to play in where you can get something that works in very little (set TERM to a dumber terminal than rxvt, dump any escape sequences your terminal doesn't yet understand to a log, and run the apps you need, then iterate...), and build up in whichever direction you want to something quite usable very quickly. Then you can spend a lifetime polishing quirky little details nobody with you will ever care about... ;)
See a lengthier reply elsewhere in this thread, re: motivations etc. but it's almost all from scratch. Frankly, a simple terminal is not a huge amount of work, and you can get something working "well enough" to start with with very little code.
The huge amount of work comes from nailing all the quirks you will have to deal with if you let a bunch of other users loose on it, with their expectations of running all kinds of applications I've never tested that does weird stuff.
This right here. Emacs actually supports graphical output (X11, Cocoa, Win32). Additional bonuses include: Esc is actually Esc; you can bind Ctrl+Shift or Cmd; you can use more than one font at a time; you can display a picture; change the shape of the cursor... It's like a time machine that takes you 20 years forward - into 1989.
There is, however, considerable upside in having all the same key bindings whether you're in local UI (maybe using tramp) or ssh to a remote server. Tramp has limitations and is a bit clunky so I use it rarely.
Clipboard integration and mouse cursor support are the two reasons I generally break out emacs-ui (my shell script for it), otherwise it's emacs-nox, generally.
Yep. I just tune the 16-color palette in Terminology (it's an INI file) to my preferred theme for the same styling in the console and terminal applications (vim).
Yah, I do pretty much the same thing. The default colours are far too dark for the dark background, so I just increase the luminance of all the colours.
I suspect the author would have had a better time had they known to do "export COLORTERM=truecolor". This is mentioned in a footnote in the linked Emacs document about detecting truecolor support, and it appears in their Konsole screenshot, but I suspect it got lost inside their layers of terminal multiplexers (as, I find, it often does). It lets you skip past all this hard stuff straight to the part where the app understands that you want true color. A terminfo database isn't even required.
Agreed - COLORTERM has its own problem. I prefer your approach, it looks very sound!
A fork of terminfo may be needed if the description of modern terminal capabilities can't be added -- or if old and deprecated attributes repurposed for that job (like in your padding example): if you're automating the correction/creation of terminfos in ~/, IMHO, it may be better to piggyback on tic as much as possible.
Anyway, to backport modern terminal descriptions to legacy programs, creating correct binary terminfos in ~/.terminfo seems the best practice. You can also invent new TERM. When I wanted to have italics etc about everywhere, personally that's just what I did for sixel-tmux: https://github.com/csdvrx/sixel-tmux/?tab=readme-ov-file#ste... : just declare a new $TERM you know to be right, and use that in the apps that let you use a little logic in their configuration file
I do that in my .vimrc:
" If Vim doesn't know the escape codes to switch to italic
let &t_ZH="\e[3m"
let &t_ZR="\e[23m"
" Italics pseudo-auto toggle: force italics if we recognize it's supported from TERM
if match($TERM, "xterm-256color-italic")==0
highlight Comment cterm=italic
endif
Another idea: have you considered that the "program that would be run by .bashrc" to generate the "unambiguous 24-bit color capabilities" could set environment variables to communicate them? Like, if you can't stuff them in terminfo, stuff them in environment variables! Env var work well on Windows, and the configuration files of terminal programs could just use these when available to override.
It's be like adding friends to your proposed TERMVERSION: TERMSETF24 etc where you've have a finer control than COLORTERM
I have tried to love TRAMP, but I can't figure out how to get eglot to work properly. I use it for C++ in a large codebase, so eglot is mandatory.
The other thing is that I like to close my laptop, reopen it the next day, and have access to the same remote buffers. (Maybe this is possible with TRAMP.)
Ultimately I just use emacs in tmux over ssh, with CLICOLOR defined appropriately, and this gets me 95% of what I want.
I wish I could get rid of those last <50ms of keypress latency. And variable-sized fonts for eglot inlay type annotations. They are quite distracting when they're the same size as the code.
TRAMP remote buffers aren’t a huge issue on disconnection because the buffer contents are stored locally: TRAMP’s magic mostly occurs on open and save. Unfortunately, your modew like eglot have to be TRAMP-aware and execute over a ssh connection somehow. Many LSP servers use stdio communication, so it should be possible to put together a mode hook that detects if the buffer file name is a TRAMP file name and then does something like `ssh remotehost -- cd /project/root ‘&&’ clangd --stdio`
Eglot does that automatically though. It detects when you're in TRAMP and launches the language server in a remote. Of course, it assumes that the language server exists at the remote, but that should be the case.
COLORTERM is also a very weird thing. Most of the time I don't need it to be defined, but when using a screen session it suddenly looks very crappy when COLORTERM is set to truecolor. Removing the env var makes it look all normal again.
Tmux supports 24-bit color. I bit the bullet and switched a few years ago, and my muscle memory remains exactly the same. I'm incredibly happy with the power, flexibility, configurability, and reliability of Tmux.
If unsetting COLORTERM fixes the display problem then it is a program issuing 24bit colors but the terminal (in your case screen) not understanding the escape sequences.
I want to make sure that people understand that you do not have to use a terminal to interact with Emacs: instead, you can use one of Emacs's "graphical" front ends for MacOS, Windows, X.org or Wayland.
Visually, graphical Emacs looks so much like Emacs running in a terminal that if the tool bar has been turned off (and most users decide to turn it off) most non-users of Emacs will assume (incorrectly) that a graphical Emacs window they see running on a colleague's computer is a terminal window (unless perhaps the window is currently displaying an image). (Graphical Emacs is an image viewer among other things.)
In a survey of Emacs users, 80% said that they use one of the graphical front ends.
Counterpoint: If you want a terminal w/o the bloat of a full GUI, but still most of the advantages, `xterm-mouse-mode` will get you 80% of the way there:
(require 'mouse)
(xterm-mouse-mode t)
(mouse-wheel-mode t)
(setq mouse-wheel-scroll-amount '(1 ((shift) . 1))) ;; scroll one line at a time
(setq mouse-wheel-progressive-speed nil) ;; don't accelerate scrolling
(setq mouse-wheel-follow-mouse 't) ;; scroll window under mouse
(setq scroll-step 1) ;; scroll smoothly
Also, I should point out that I use `(setq auto-save-default nil)` - as God intended.
People are going to be using Emacs with a monitor. It's more bloat to have to have a GUI application that emulates a terminal and then run Emacs inside that emulator instead of just having Emacs be the GUI itself.
Unsaid in all of this is that Emacs isn't my main text editor - that is the job of BBEdit where the GUI is both expected and rather good - instead, it is the "editor of last resort" (Vim is inscrutable to me).
The emacs GUI inside a terminal window - invoked w/ xterm-mouse-mode - is still rather minimal. I'm thinking more about emacs-gtk or Aquamacs. You could argue that the GUI gives you access to more system capabilities in a straightforward way - buttons/controls v keybindings - but this isn't a dealbreaker for me.
As long as I have a readily available reference for keybindings, that's good enough for me.
I’m not sure what your point is, exactly. If you run Emacs in a terminal, you can press F10 to access the menus. They’re the exact same menus as you would see in the Emacs GUI.
There’s just no way that the GUI adds bloat in any meaningful sense; it’s the same program either way.
To my way of thinking, the idea that you would remove the toolbar in graphical mode defeats the purpose.
Emacs keybindings are complicated enough that I'd rather keep the ability to click a button available at all times even if I don't use it regularly - see my prior comment re BBEdit.
Such is life that I have some physical issues with my hands that make a purely keyboard driven interface occasionally uncomfortable. The chorded keystrokes in Vim were not satisfying to me, so I went in an emacs direction ¯\_(ツ)_/¯
When I started building out my init.el, it became apparent that I needed an escape hatch from time to time - this is it.
I don't run emacs in a terminal emulator. Emacs is my terminal emulator. All my terminal programs get run in a (graphical) emacs shell-mode buffer. The rare program that needs full screen control uses term-mode.
It's not the most featureful or fastest terminal emulator, but I can jump around the buffer like any other buffer and seamlessly access it along with the other content I'm editing.
On a remote computer where I can't run a graphical emacs, I either remotely edit files via tramp, or run a headless emacs server process and connect remotely with a graphical emacsclient.
I also use emacs graphically, somehow I managed to switch after years of using it in the terminal and now I prefer it. However, I've never managed to settle on any of the terminal emulation modes. Just too many keyboard shortcut conflicts and complicated magic to remember how to copy and paste between different types of buffers.
These days I've pretty much settled on having and emacs window and GNOME Terminal side by side as my two main work windows, and it's pretty comfortable. I like that I can mouse select and copy from the terminal window and then C-y into emacs and vice versa.
If I really have some need I might still open a shell or ansi-term session but it's pretty rare now. Sometimes it's convenient when I find myself in a remote shell session and want to have a text editor and shell vertically tiled, I'll just run emacs with ansiterm because for the life of me I can't ever remember the screen keyboard shortcuts for splitting windows.
It's been a while since I've run with that configuration, but setting the variable server-use-tcp to non-nil enables it. On startup, the server creates a file with the connection details that emacsclient can consume. I think it just works if the filesystem is shared between client and server machines but command line options allows the use of a copy on the client.
Good point. I never considered that people might think I run Emacs in a terminal. Graphical Emacs is a lot better, but it's still great that 90% of stuff works and looks the same in a terminal.
I agree. I use X based emacs and the protocol is efficient. The experience seems as lightweight as using a terminal, and NOT like using VNC or similar.
advantages include fonts (even different ones at the same time), menus, mouse, colors, multiple frames/windows, and more...
You can also run the graphics locally use tramp to access remote systems. I can edit config files on a remote linux-based router which basically has busybox, with all the bells and whistles.
I've always drifted back to the terminal based emacs for two reasons.
While I've been using X11 off and on since the 90s, I still do not understand how to deal with fonts beyond helplessly googling for a constructive command I eventually stumble on something that works and then I have to leave it alone.
The display performance of most terminal programs is still noticeably faster than the X11 setup I end up with in a corporate environment.
The font stuff is 50x better than it used to be and there's even a reasonable font selector built right into the GUI with the "Set Default Font" in the context menu (ctrl-right-click). And then you can "save" that from the GUI and get a line like this in your .emacs:
'(default ((t (:family "FantasqueSansM Nerd Font Mono" :foundry "PfEd" :slant normal :weight regular :height 98 :width normal)))))
Right. There's also the sophomorically named "TRAMP" for connecting from a locally hosted emacs session to files across a wide range of protocols including ssh.
TRAMP also exhibits a wide range of performance problems. Getting it not to hang the display of the buffer list or lock up Emacs completely for minutes at a time requires a deep dive in to a decade of Stack Overflow breadcrumbs. I've done this, and I'm still not completely satisfied with the performance. I do however love the concept.
No, it requires only one thing: turn on persistent SSH connections. This means that new SSH connections to the same server do not actually require opening a new TCP connection, do not require negotiating new session keys, and do not require additional authentication. To turn this on, add these settings to your ~/.ssh/config file:
ControlMaster auto
ControlPersist yes
ControlPath ~/.ssh/control/%C
You should also use key–based authentication so that TRAMP never has to ask for a password, but this is less important once you are using persistent connections. Add something like this to your ssh config:
Host orod-na-thon
Hostname 192.168.2.133
User db48x
IdentityFile ~/.ssh/id_ed25519_your_private_key
Now go to Emacs and open up /ssh:orod-na-thon:~/.ssh/authorized_keys. This is the TRAMP path for your authorized_keys file on the machine named orod-na-thon. It will ask you for your password. Paste in your public key (which you can get by opening up the .pub file that goes alongside the private key in ~/ssh), and save it. Now future connections to orod-na-thon will be able to authenticate automatically using this key pair, and there will be no password prompt when you open a file there.
One problem that I looked briefly into, but have not found a solution is that often I'm browsing the file system (in a terminal) on the remote box and want to open up a file in a certain location (often just for a quick edit). The process of typing pwd, and copying the location into emacs (similar applies to Neovim) already is too much friction, that it's easier to just open the barebones Vim on the remote.
Anyone aware of a solution of how to start the local editor from the remote to open the remote file?
Best solution to that is to use Eshell. Run `M-x eshell` to open the shell inside Emacs (on your local machine), then type `cd /ssh:orod-na-thon:path/to/files`. This will transparently ssh to the server and change to the path to your files. The `ls` command will now show the files that exist on the remote server rather than the local computer. You might now expect that there would be an eshell command that you could type to open a remote file in your local Emacs, but there isn’t.
No, in Emacs you always open a file by typing `C-x C-f`. (Unless you rebound it to some other key, in which case make the obvious substitution.)
Any time you hit `C-x C-f` to open a file it defaults to opening files from the working directory of the current buffer. The working directory of the eshell buffer is on the remote server, so the default list of files that you see are all the ones you were already looking at with `ls`. You can start typing a filename and autocomplete will do the rest.
This is an even deeper and more convenient composition of the shell and the editor than having special commands for tasks like opening a remote file in the local editor.
> You might now expect that there would be an eshell command that you could type to open a remote file in your local Emacs, but there isn’t.
No, in Emacs you always open a file by typing `C-x C-f`
`C-x C-f` calls the `find-file` function which can be called directly from eshell
Technically correct, but it is way easier to hit `C-x C-f` just like you would when you are opening a file at any other time. No need for special cases, you just open the file.
Try some variant of this in your remote .bash_profile or what-not:
if [ "$PS1" ]; then
export PS1='\h:\w\$ '
if [[ "x${TERM}" = "xeterm" || "x${TERM}" = "xeterm-color" ]]; then
function set-eterm-dir {
echo -e "\033AnSiTu" $(whoami)
echo -e "\033AnSiTc" $(pwd)
echo -e "\033AnSiTh" $(hostname -f)
}
PROMPT_COMMAND=set-eterm-dir
fi
fi
Now, locally, invoke M-x ansi-term, then within the ansi-term ssh to the remote machine. Change dirs, hit C-x C-f to open a file in the current (remote) directory.
Apologies, I meant to add more saying I could not get a remote machine to invoke the local emacs reliably. I got close with hacking up emacsclient to ssh back, but was eventually stymied by new network policies at $JOB and eventually settled for the above.
In the specific case of Emacs, assuming you're connected via ssh, sshd is running on port 22 of both hosts, an Emacs server is running locally, and emacsclient is in the default local path, a bash function or script along the lines of
for i in "$@"; do
ssh "${SSH_CONNECTION%% *}" emacsclient --no-wait "/ssh:$(hostname):$(realpath "$i")"
done
should work.
For best results, use ssh-agent + agent forwarding to avoid password prompts.
> The process of typing pwd, and copying the location into emacs (similar applies to Neovim) already is too much friction, that it's easier to just open the barebones Vim on the remote.
Vim has had the ability to open files remotely over ssh, sftp, etc. for a long time using the built-in plugin netwr [1].
Neovim takes it to another level [2]:
Nvim's RPC functionality allows clients to programmatically control Nvim.
Nvim itself takes command-line arguments that cause it to become a client to
another Nvim running as a server. These arguments match those provided by
Vim's clientserver option.
Yes, that's great, and it makes things extremely snappy while the socket connection remains open. Sleep your computer overnight, experience some network stormy weather, leave Emacs buffers open for weeks (as one does) and eventually one of those ControlMaster sockets are going to become wedged, and when that happens you're going to go through some things.
If that happens and you don’t want to suffer through the default 30–second (or whatever it is) timeout, then shorten it. Add `ConnectTimeout 5` to your config.
Good idea fine tuning the timeouts. I took your suggestion and dropped the SSH ConnectTimeout to 5, I was still using the default. I also peeked at the Tramp timeouts, and I realized that I had the variable as tramp-connect-timeout and not tramp-connection-timeout. Thanks!
I had this trouble when I upgraded emacs. I'm now stuck on 27.2 because later versions broke tramp in some way, and I haven't taken the time to debug it.
> [1]: You can type ^[ (escape) by pressing CTRL-'['. Don't type ^ and [ as separate chars, that won't work.
You can also use the \x1b escape code(or the \e alias) together with a command that accepts escape code in order to avoid nonprintable characters on the command-line:
echo -e '\e[48;2;255;0;0mHello\e[0m'
For scripts, always use printf though since echo -e is nonstandard and even plain echo differs in edge cases:
The great thing about running everything in a terminal is you can copy anything that is shown in the terminal. Frustrating when a GUI app won't let you copy some text it's showing and you have to retype it somewhere else by hand
Pasting into a terminal is such a minefield, though.
On net I'm not sure it's an improvement. Being a pure ncurses-esque console app limits the richness of what you're able to display. Native controls and dialogs are in fact a good thing and not a cardinal sin.
this 10,000,000,000%!! (dr stone ref.) I love modal editing and vim a ton, but I couldn't stand the half baked solutions terminal editors gave me. We have an entire rich graphical platform, why not use it?
This depends heavily on your workflow. For example I use vim almost exclusively for editing files, and drop back into the shell to use external tools (e.g., git, linters, formatters, debuggers, file tree manipulation, etc.) So my terminal emulator is already the main application. For this workflow the lack of a dedicated GUI is almost immaterial.
Really shameful that they lock away standards under such high prices that people have to go to such lengths to not buy them, with potential results like this. If it's a standard it should be available. It maybe made sense to charge a printing fee back when they were only available as books but I can't believe a crappy pdf was $200.
I wrote the patch for 24 bit color in xterm, but I have to admit I was today years old when I learned how to make ncurses work with xterm-direct. I was already using my own terminal library in Lisp, but it's still useful to be able to fall back to using ncurses. Thanks to Mr. Dickey for answering a stack overflow question, and the author Chad for bringing it up.
That's a very interesting investigation. However, perhaps computing has lost its way, when a production system uses emulators of 1970s terminals to run a 1970s editor on a version of a 1970s operating system. I really think that we're stuck in a local maximum and there are probably much better ways to do things. Our computers are millions of times more powerful now, but we're still doing things the same way as literally half a century ago.
People may be stuck in what they think is a local maximum. I ditched emacs close to 20 years ago (for jEdit, later Sublime and finally IntelliJ) and have never regretted it in the slightest.
I don't want my editor to read usenet or play solitaire. I want it to edit text and surface useful contextual information, and otherwise stay out of the way and let me devote my mental energy to the task at hand and not futzing with the damn editor config.
I really want to use Orgmode (or maybe something like Neorg/Vimviki for vim), but the biggest issue with all of those things is that somehow, there still isn't a proper mobile app for any of them. No, I don't want a simple TODO app that uses Orgmode under the hood. I want an actual app for Orgmode, something which works like Obsidian or Logseq. (In fact, Logseq supports orgmode, but that support is very limited and sometimes incompatible with actual Orgmode)
I don't understand how there are so many passionate users of Orgmode yet apparently not a single one of them is a mobile developer who sometimes takes notes on their phone.
By contextual data I’m really just talking about things like being able to auto complete from it only the code base, but associated sql schemas as well. Or presenting code navigation as a sorted hierarchy of classes, functions, etc. Lazy human want computer do work.
What he wants is to connect his editor to an SQL database and have TAB completion in a string for keys and tables in that database. He wants to be able to traverse a codebase by classes rather than by file. Basically he wants Smalltalk.
> I ditched emacs close to 20 years ago (for jEdit, later Sublime and finally IntelliJ) and have never regretted it in the slightest.
In that time I've carried on using Emacs, and have regretted it several times ;)
However, Emacs has come on a lot. It's much easier to configure than it used to be, and LSP has brought its semantic features back into the range of other editors. Its text processing abilities have always been stellar and far outshone anything else I've tried.
I'm very happy I stuck with it. I've kept many years of muscle memory and tricks and avoided having to reconfigure 3 different editors. Plus there are killer features like magit and org-mode.
That timeline is very enlightening and those references are extremely useful. Must have taken a lot of time to find and make sense of all that material. I'm especially thankful for the explanation of terminfo files.
> We used semicolon (like other SGR parameters) for separating the R/G/B values in the escape sequence, since a copy of ITU T.416 (ISO-8613-6) which presumably clarified the use of colon for this feature was costly.
Non-free standards are self-defeating. What's the point of the standard if the people who need them to implement stuff can't read them?
I tried using kitty recently. Its really neat, until you try to sshing somewhere. If the remote system doesn't have kitty in its terminfo, nothing works. Kitty's "solution" to this is to use a custom ssh wrapper, which I'm sorry, is not an acceptable solution. A simple solution would be to let users override what TERM gets set to, but they refuse to allow this on some moral ground.
Maybe I'll try it again in a few years when its in more terminfo packages.
I just type 'export TERM=xterm' whenever I log onto one of these systems (I use tmux-direct as my $TERM). It seems like Debian 12 has a lot of newer terminfos these days in the default install, and I've tried to upgrade most places I ssh to that. Like our CI system can take a Docker image as the place where your code runs; so instead of using the default one from 2018 or whatever, I just made my own that has all these niceties.
(I use kitty on Mac and Windows Terminal on Windows, but it ends up not mattering because I'm always in tmux, so that's the terminfo that remote machines need. I never bothered setting up 24 bit color though, so I have no idea if that would work with my setup. I switched to TERM=tmux-direct to get italic fonts in Emacs running on Linux inside tmux inside an SSH session from Windows Terminal. It works!)
Yeah that's silly. TERM is as broken as web user-agents. You just have to accept that and `export TERM='xterm-compatible, like xterm-256color, like gnome-terminal'` or whatever :-D
The distros I use (debian, arch) have a kitty-terminfo package. First thing I install when I ssh into a machine. My terminal settings just work and I don't have to do archeology on decades of obscure terminal escape sequences.
Servers in workplaces are often ephemeral. It's easy enough to write provisioning code that updates the system terminfo database, but it needs to be written, tested, reviewed, and deployed. Speaking of deploys, how are your Q1 OKRs shaping up?
That is what kitty's "custom SSH wrapper" does, if I understand correctly. Each time you ssh into a server, it will copy kitty's terminfo file if it doesn't already exist.
I run emacs with minimal colour, so will probably never use this but I very much enjoyed the read. I'm rather used to the garish green of tmux and a small amount of rusty red brown from the amounts of emacs colour left in my life.
One of the most frustrating things about terminal Emacs is figuring out how to get nice colors in tmux. By the time I get it working finally, I have no idea which combination of the tens of things I did made it work.
I also really really want to use TRAMP by default but there are some issues that keep it from being painless.
The problem with getting lsp modes working in things like docker containers is:
- docker containers set PATH
- tramp doesn't use PATH by default
- even using `(add-to-list 'tramp-remote-path 'tramp-own-remote-path)` doesn't work
tramp-own-remote-path doesn't work because:
*Note* that this works only if your remote ‘/bin/sh’ shell supports
the login argument ‘-l’.
For whatever reason this doesn't work in many docker containers or perhaps is a NixOS issue... not totally sure.
I need to revisit this at some point, because otherwise you have to update your tramp connections every time the PATH in the docker file changes, which for many make it a non-starter.
I'm sympathetic to the security first approach of why tramp doesn't just use PATH or the behavior of `tramp-own-remote-path` by default, but it makes working with the insecure by default rest of the IT ecosystem more difficult.
Outlive right? Session handling is one thing I recall being the best feature of tmux/emacs.
The solutions are better these days for session handling in emacs with save-place-mode and one of:
persp-mode
bufler (using bookmarks)
However one of the biggest advantages for me is also having things like eshell or vterm get restored. That's arguably a less emacs friendly way of doing things than say using a compile buffer or async-shell-command with proper history, but... can't change over everything all at once :)
My guess is that widespread 24-bit support in terminal apps is still held back by the fact that the macOS terminal still doesn't support it. As far as I can tell nearly everything else does :(
It seems possible to offer graceful degradation for non-supporting terminals, and I'd be surprised if the number of Terminal.app users that would even notice the difference would be that high:
Surely everybody who cares about a good terminal experience uses some third-party client already? Then again, maybe I'm underestimating managed/corporate devices that don't allow third-party terminal emulators, since they do have quite critical access in the end.
> Surely everybody who cares about a good terminal experience uses some third-party client already?
Unclear. The built-in Terminal app is pretty good otherwise, and true color isn't necessary for a lot (most?) of common terminal usage. It's not so poor that all command line users would definitely have a third party app.
IME, this is like the golden age of terminal apps in general and macOS-compatible ones in particular. There are several really good terminals for macOS:
Protesilius has changed the game on theming, to the extent that modus-operandi and modus-vivendi are now baked into emacs.
They are fucking incredible and you only need to look at their readme to see the the intense duty of care. It’s not just aesthetics it’s accessibility.
My blog is built by emacs and I use modus-operandi for syntax highlighting.
Hmm... My Emacs has just worked with full colour in terminal for years. I didn't know it was a thing that needed to be configured. Skimming through the article I hope I never have to know this stuff.
I also have icons in my terminal by using nerd icons.
The article is a deep dive on the evolution on terminal color, mentioning some of the "colorful" characters in the scene. This is specifically about the ultimate 24-bit expression of gawdy terminal bling, aka COLORTERM=truecolor. Perhaps you're not the kind of person who would notice if their Emacs color theme looked wrong because the theme designer's carefully chosen 24-bit color was mapped to the nearest color in the 256 color palette of the stock terminal.
Oh, I am that kind of person, believe me. Mine looks identical in the terminal. I use zenburn theme. The only thing I ever set in my bashrc is `TERM=xterm-256color`. I'm not sure why. I do have `COLORTERM=truecolor` set, but I didn't do it myself. It just works for me with or without those variables set in both xterm and alacritty.
I really love emacs in the terminal. I created a docker image that looks beautiful (and identical) whether you're editing locally or remotely. It's configured properly for 24 bit color and unicode, and it also has all the underlying binaries for native compilation, lsp, ripgrep, fast json and so on, all ready for both amd and arm. Caveat -- I haven't used it on windows yet. https://hub.docker.com/r/wwarner/emacs-gopy
This is a cool deep dive into terminal standards and their footguns but... not sure why?
Unless you're showing pictures in your terminal, why would you want true color and the overhead that introduces? If you're just doing a status bar on a black background, your eye won't really see the difference between FF0000 and FF1515 (which I picked because the next 16 bit color is FF3333 which you can already barely tell the difference from).
I'm just saying I'm not sure I would want that much color choice, it would just give me anxiety about picking the right color!
> I'm just saying I'm not sure I would want that much color choice, it would just give me anxiety about picking the right color!
Developers of themes for Emacs often target the GUI first, where 24-bit color is the default. If you're a hardcore terminal Emacs user (and there are many legit reasons for this) you're stuck by default with 256 colors, and the themes just don't look right due to having to map the carefully chosen colors to the nearest color in the 256 color palette.
Exactly correct. Loading solarized in 256-color emacs shows just how poorly the (carefully-selected) Solarized colors map onto the xterm-256color palette.
It clearly depends on what you do but having more colors is nice.
Even if 256 colors are available it's not enough. The palette of the 256 colors is gruesome. You can clearly see that a technical person without any knowledge of color theory designed it.
Thank you for this article. One of the great frustrations of using Magit for Emacs is the inability to read things because the foreground and background colors for various things end up the same on some platforms.
From this article I found the command "toe -a" and was able to try various TERM values until I found one that makes Magit readable: vt100. This turns off colors entirely, which may not be pretty, but at least I can read everything now.
In my case, I don't care. Give me plain black and white all day long--and I do.
When you have that many colors on the screen it no longer is helpful and only becomes a blur of clown colors.
If you must, use just a couple of colors--or even just shades of gray or bold--to highlight important things like functions.
But that's what indentation is for. You indent/format your code to help find things. It seems to work well in books. And that works everywhere without any work involved to make it happen.
> It turns out, for years, popular terminals have supported 24-bit color. And yet they’re rarely used.
Hard to say how rarely. I've been using 24-bit color in bash prompts and neovim scheme for a while. Can't go back to 16 or 256 color ones, they are just too limited.
I wish tty terminal also supported 24-bit color.
For setting the color in bash I use something like:
If only there was a way to magically pass extended modifier keys to Emacs over a terminal…
It's the only thing stopping me from using Emacs remotely over a terminal: I really do need my Super modifier. Hyper I can live without, but being able to do shift-super-f for example is fundamental.
Interesting, as a Vim user I'm surprised Emacs doesn't have better theme support. I always assumed it had everything. In modern Vim there's even hex support.
I got that impression from the article because of the length he went with ANSI escape codes. I wouldn't touch that other than for my PS1. But I see now there are sane options and the author probably just wanted some fun.
I enjoyed tinkering for years. One day my Vim colors wouldn’t work in Tmux for like the 5th time in my life. I said, you know what? I’m done with all of this.
I appreciate that what you did is fun. I just thought I’d share a perspective that sometimes we’re sick of tinkering.
EDIT: for iTerm+zsh, I collected some functions for setting tab colors: https://gist.github.com/aclarknexient/84ebe33c1879f921685304...
EDIT2: Contrast terminfo with the old-school method of connecting to HTTP or SMTP with telnet. Yeah, it's simple text but that text is being displayed by a complex application that supports all sorts of wild stuff!