What would a clean-slate implementation of a serial terminal look like today?
I work in embedded, and a serial terminal is practically a must in any system, if only for debug. A way to provide input and receive output.
You'll definitely want a FIFOs for RX/TX. You also likely want control chars and a way to shortcut the FIFO, if only to interrupt output. The FIFO means you can abstract away the exact physical communication medium.
TUI interfaces mean you also want a way to tell the program what the dimensions of your screen are, through the control chars.
I've given this some thought. The way to improve the TTY is to give it channels, like SSH or RDP. This way you can transit unrelated data over the same connection. And, if you think about it, most of the complexity of a serial interface is handling mode settings and state; streams abstract that away for you. Streams get rid of the need for control characters and in band signalling. Streams could be given priority and interrupt levels.
A big fat YES from me. I have been looking into TUIs lately, and the state of terminal handling is ridiculously bogged down by years of cruft that, honestly, I don't give a shit about anymore. To put it succinctly: I want Alt-Home and Ctrl-Click to work on a TUI. Period.
If I had superpowers, I would layer the system as a very low-level library that recognizes keyboard / mouse / tablet / microphone input hardware events and encodes them somehow (so Alt is a separate key, pressing multiple keys is handled, clicking + moving the mouse is handled, etc); then a protocol that can give you this event stream, either from a local or remote source; then a layer that maps this into higher-level events (so Alt and Home pressed together is interpreted as Alt-Home). The idea would be to have the ability to detect, even over a serial connection, any of these input events, without resorting to escape codes.
I don't even know if this all makes sense; I just hate the loops you have to go through today to even try to make sense of key combinations.
This might not be accurate. RS-485 is an electrical signaling standard for carrying arbitrary binary signals. It's usually used for carrying UART data, as a longer-range / higher-speed replacement for RS-232. It has no opinion about what is transmitted, and has no concept of channels or even any particular encoding format.
How does that make it an RS-232 with channels? You’re confused. RS-485 just like RS-232 is an electrical specification for serial signaling (with multiple access), nothing more nor less. There are USB to 485 converters that basically operate exactly the same as their serial port counterparts and will show up as a tty device in Linux/Unix flavor or a COM port in Windows.
It's a bus, rather than point-to-point. Schemes to address devices on the bus separately exist. You could call them "channels" even. RS-232 can't do that, as it isn't an electrical bus. Respectfully, you're confused.
Since tty's are fairly well married to a serial port, adding an ioctl() (or similar) that would switch channels would seem likely. RS-485 would provide a way for that to work not just with ptys, but with real physical devices as well. With reasonable backward compatibility.
RS-485 is an electrical signaling standard, for carrying arbitrary bytes between a transmitter and a receiver. There isn't even any built-in concept of a packet, let alone an addressing scheme. It's no different than RS-232 at all, in that respect.
When compared against RS-232, only the signaling voltage and impedance requirements are different. That, and also RS-485 has no standardized connectors or side-band signals (like CTS/RTS).
Sometimes RS-485 is used half-duplex over a shared differential pair. In situations like that, slave devices are often designed to keep their transmitters disabled unless responding. Sometimes, these setups will have a couple of devices listening for data.
Lots of people implement custom bus protocols _over_ RS-485. But RS-485 by itself has no concept of buses, packets, addressing, channels, slave devices, or master devices. It's just an electrical signaling standard. Nothing more, nothing less.
It's kind of like saying "AM radio-waves are a format for carrying music". Sure, some people transmit music over AM radio. But that doesn't mean that the actual radio waves have any knowledge or accommodation for what they're carrying.
The way 485 addresses devices is... just to prepend an address byte. Or bytes. Whatever you want, really. The link layer, which RS-485 is, doesn't care.
As others have said, RS-485 isn't really the way to do this. It looks like RS-232, but it's just close enough to mislead you while being far enough away to be a total nightmare. For example, any bus collisions have to be handled in software. A driver that has to recognize and deal with bus collisions looks very different from one for a dedicated line.
This rabbit hole is probably not worth going down.
This is really reaching. Channelization as being referred to upthread in reference to SSH or RDP is a protocol level abstraction, rs485 bussing is at the electrical level for multiple physical devices and also doesn’t specify any addressing scheme, that would be specified as a higher level protocol (that could be done just as well over rs-232). Device addressing and channelization are not even logically the same thing.
Ethernet as it is typically deployed is not a bus either but it is used in channelized communication (packet switched no less) just fine.
Edit: responding to your edit. The point still stands this is orthogonal to the physical or electrical layer. The vast vast majority of the internet including Ethernet and fiber links are point to point and clearly they carry a tremendous number of separate streams, and a small amount of broadcast. You can run layer 2 over an RS-232C serial port, whether its Ethernet, PPP, SLIP or ATM. In Linux PPP is in fact implemented as a tty line discipline.
As for RS-485, there are widely used standards such as DMX512. It is not a strongly supported idea to put this stuff in the kernel, when performance in user space is sufficient.
Finally, RS-232 can do the same thing (multidrop) with electrical repeaters. See MAX3322 eg.
Clean slate definitely object/packet based - just because we need to find the dimensions of the screen doesn't mean it needs to be done via control characters.
Fairly easy, a few thousand lines of C. Bit of work, though. Parser is the hardest, a lot of details to get right. You don't really need a buffer for the output, because you can't type that fast.
Of course, driving the display is a bit harder. I'm using a parallel tft, wrote the driver, too. Keyboard code is kludgy because I'm not using diodes for cost reduction on the bom.
You can see it here: violence.works yes that's a url.
I'm both amazed at what you've accomplished and a little miffed that someone beat me to it: I've been working on-and-off on a similar project for a couple of years now haha. Awesome work
What display did you end up settling on, where'd you source it, and how many columns do you end up fitting? That was the biggest sticking point for me.
Are you talking about a physical machine or a protocol design? I don't think you can abstract away the exact physical communication medium from a physical machine (or your cable won't fit in the hole, or it'll burn out your circuitry), and the protocol design probably doesn't need a FIFO.
Protocol design: maybe typewriter text and shitty VT100 escape sequences were a bad idea? Let's use HTML-style tags, but in an endless tag soup similar to VT100 semantics, not a tree.
But we have 55 years of experience with the advantages and disadvantages of the ADM3A, VT100, and similar terminals, and we've gradually improved it over that time. Now it's ten thousand times faster than a physical VT100, supports 24-bit color, auto-highlights URLs, has enormous scrollback, supports copy and paste, allows applications to handle mouse events, can be recorded with script or ttyrec or asciinema, can be detached and reattached later with screen or tmux or mosh, can get local echo with mosh, can share a screen and keyboard between people in different locations with screen or tmux, and has hundreds of thousands of programs already written for it — aside from backward-compatibility with teletype software like ex, ls, sh, grep, ps, ping, vmstat, and so on, you have user interfaces designed for character-cell terminals like Emacs, Vim, links, mtr, dstat, top, htop, dialog/whiptail, less, mutt, irssi, ircII, weechat, nethack, sl, robotfindskitten, command-line editing in things like tcsh and bash, progress bars in things like apt, nano, and of course the aa demo bb (if run with env -u DISPLAY).
There are a lot of things that can't really be done without breaking compatibility with that ecosystem. Proportional fonts of continuously varying sizes, or padding? Now screen positions are no longer integer multiples of a character size, and you can't get things to vertically align by inserting space characters. (This is already a problem with Unicode, where different monospace fonts are monospace over different amounts of the character set; see http://canonical.org/~kragen/sw/dev3/randomlayout.py for some attempts to test the limits.) Pixel graphics? Are those going to pass through tmux unharmed? (Some recent attempts to jimmy pixel graphics into terminals won't even work over ssh.) How will copy and paste work with them? Will they send another megabyte over my cellphone connection every time I press ^L to redraw the screen?
We can definitely do better. But doing better is going to require a lot of experimentation to be a Pareto improvement over 55 years of tweaks to the VT100. Most of the ideas we come up with are going to turn out not to work as well in practice as we hoped they would in theory.
And not just the terminal. Why on Earth people thought that the regular PC keyboard and the mouse required anything more complex than the good old RS-232.
> Teletypes were once connected across the world in a large network, called Telex, which was used for transferring commercial telegrams, but the teletypes weren't connected to any computers yet.
Telex was a private system disjoint from the telegram system which was, in most countries, run by the PTT (government agency called Post, Telegraph and Telephone). In the US and some other countries it was private, of course, where Western Union was the dominant player.
I remember staying in a hotel in India the early 80s that had four different systems (with its own model of printing terminals, each one a clunky electromechanical beast). One was telex and one some iron curtain alternative, I remember, but I don’t remember the other two.
> ... a backspace key is often useful. This could of course be implemented by the applications themselves, but in accordance with the UNIX design philosophy, applications should be kept as simple as possible. So as a convenience, the operating system provides an editing buffer and some rudimentary editing commands
This approach far predates Unix, and Unix got it from Multics. Mainframes implemented it in their channel controllers to save time on the CPU (it was central, after all), while minicomputers typically did it the way Unix did.
Non-“glass tty” video terminals didn’t have frame buffers in the sense today, but just character storage. Characters were rendered by looking up pixel data in a ROM (before that, hardwared!) which might only have been a rectangle of 32 pixels or less.
> In a similar way, it is possible to configure the TTY to react on a stop flow byte and a start flow byte. These are typically ^S (ASCII code 19) and ^Q (ASCII code 17) respectively. Old hardware terminals transmit these bytes automatically, and expect the operating system to regulate its flow of data accordingly. This is called flow control, and it's the reason why your xterm sometimes appears to lock up when you accidentally press ^S.
I don’t remember any terminals that recognised soft (inline) flow control though they may have existed. Instead, they signaled it through the transmission cable itself. If you only used three wires in your serial cable this was impossible, but check out the full pin out of a DB-25 or DE-9 (sometimes ignorantly referred to as “DB9”) connector.
Oh yeah, I remember that this is where I really understood line discipline. For example, why backspace/Ctrl-H worked even if you are typing in things directly to a terminal (e.g. cat >> output file) and it did not need an editor to "interpret" the key, whereas things like Ctrl+A would need an editor. I know it's pretty basic, but this is the article that really helped me understand it.
Sorry, I am still mystified. Especially with software flow control. Sometimes using emacs "search-forward" everything just stops and it takes time to figure out what just happened. But this phenomenon has becomen increasingly rare in recent years. It is truly mystifying.
Its amazing to me that despite our enormously powerful PCs and wonderful modern software tools we still rely on foundations built in a different time tty, awk, even bash scripts.
Half the time its great to work with such finely tuned Darwinian survivors, the other half I want to start again. Doing helm k8s yaml templates makes me wonder if its possible for us to write something simple and clean that could replace those old building blocks.
I won't defend TTY or bash, but I will say that awk has stood the test of time because it's an amazingly potent tool for doing ad-hoc processing of textual information, which is still a frequent task even in the era of Kubernetes and yaml templates. Why does it need replacing?
When I learned awk I felt like I unlocked a superpower.
Yeah: it's like, the problem we are trying to solve largely hasn't changed, so the solutions are maybe a bit lacking in elegance sometimes, but they are generally fine... and the concern would be that someone trying to do it today would be like "the correct way to do this is to have some ridiculously complex stack built on top of 70 pseudo-standards from Google with a single bespoke implementation in JavaScript that no one could ever reimplement".
I work in embedded, and a serial terminal is practically a must in any system, if only for debug. A way to provide input and receive output.
You'll definitely want a FIFOs for RX/TX. You also likely want control chars and a way to shortcut the FIFO, if only to interrupt output. The FIFO means you can abstract away the exact physical communication medium.
TUI interfaces mean you also want a way to tell the program what the dimensions of your screen are, through the control chars.
What else?