FWIW, I wouldn't try to parse escape sequences directly from the input bytestream -- it's easy to end up with annoying edge cases. :-/ In my experience you'll thank yourself if you can separate the logic into something like:
Without this separation, it's easier to end up with bugs where, e.g., a UTF-8 sequence or an ANSI escape sequence is treated differently if it's split between read() calls (https://bugs.chromium.org/p/chromium/issues/detail?id=212702), or invalid input isn't correctly recovered-from, etc.
> - First step (for a UTF-8-input terminal) is interpreting the input bytestream as UTF-8
> - Second step is "parsing" the scalar values by running them through the DEC parser/state machine.
Unfortunately, you may need to intermingle some logic between these two steps.
While VT100 style control sequences are usually introduced with an ESC, they can also be represented as a C1 control sequence, e.g. 0x84 instead of ESC + D, 0x9b instead of ESC + [. These sequences are raw bytes, not Unicode codepoints, and their encoding collides unpleasantly with UTF-8 continuation characters.
Since there's no standard which specifies how UTF-8 should interact with the terminal parser, you're a little bit on your own here. But probably the simplest fix is to introduce a special case into the UTF-8 decoder which allows stray continuation characters to be passed through to the DEC parser, rather than transforming them to replacement characters immediately.
:-) The UTF-8/Unix FAQ and existing terminal emulators don't agree with you here. As you say, there's no spec for this, but here's what Kuhn's FAQ says (https://www.cl.cam.ac.uk/~mgk25/unicode.html#term):
"UTF-8 still allows you to use C1 control characters such as CSI, even though UTF-8 also uses bytes in the range 0x80-0x9F. It is important to understand that a terminal emulator in UTF-8 mode must apply the UTF-8 decoder to the incoming byte stream before interpreting any control characters. C1 characters are UTF-8 decoded just like any other character above U+007F."
The existing ANSI terminal emulators that support UTF-8 input and C1 controls seem to agree on this (VTE, GNU screen, Mosh). xterm, urxvt, tmux, PuTTY, and st don't seem to support C1 controls in UTF-8 mode. So I don't think poking holes in the UTF-8 decoder is necessary, especially since allowing C1 in UTF-8 mode is rare anyway.
Note very few terminals implement UTF-8 and C1 controls and in particular xterm (which is kind of the defacto standard) doesn't because of the issues you outline. My opinion is they should just die as a legacy thing. No programs depend on them.
That's a perfectly reasonable answer too. There's a couple of other VT100 features that are safe to omit if you aren't going for full "historical accuracy" -- VT52 mode, for instance, has been obsolete for 30-40 years now.
As an aside: I wonder how useful it'd be to assemble a report documenting all known terminal control sequences and other behaviors, what terminals they're available in, and how frequently they're used in modern software. There are some big gaps between the DEC documentation, ECMA035/043/048, and actual implementations of terminal emulators.
The comments here don’t seem to reflect what I think is the most interesting point here: quick loops of satisfaction. So much of programming often takes forever to get any real utility or see progress. That can really be depressing, especially for a side project. That’s what I love about cooking or sewing; you quickly see the process come together. I wish programming was like that more often.
Yeah, I'm using my own terminal, and my own editor. The terminal also relies on a font-engine I have heavily modified (I converted the original from C to Ruby). So I "control" the whole pipeline from the editor to the actual pixels, and on one hand it has all kinds of quirks I wouldn't wish on someone else, on the other hand they all have bits and pieces that are custom-written to fit exactly what I want, and which features gets implemented are decided almost entirely based on which little change feels like it'll immediately improve my life right now (and I'm not joking - I spend enough time in front of my terminal that fixing small aspects of the terminal or my editor does feel like it is making an actual improvement in my happiness).
I'm constantly disappointed by my terminal. It always feels sluggish, especially with something like ohmyzsh. Boggles my mind why the whole pipeline is synchronous too.
It is effort, but it's also relaxing. I'll start by saying I don't think this is for everyone and it is far easier if you like minimalist environments. E.g. to me Visual Studio Code for example is an utter nightmare of visual overload. I want my editor to at most show a minimal one-line or less status area and maybe, optionally line numbers. Nothing more. I actively don't want it to pop up auto-completion options or documentation. And so on.
A lot of my requirements makes things simpler rather than harder, and that's not necessarily the case for others.
I started with the editor when I realised my Emacs config was several thousand lines and I though "I can write a whole editor in less". As it happens, a basic editor can be extremely small even in a language like C (e.g. see Antirez' "kilo"), but important: It can be extra small when you write it only for you. I can overlook bugs that doesn't bother me, and only implement functions I care about, and make assumptions about the environment others can't.
E.g. my editor presumes you're running under bspwm. It can run without it, but requires adapting helper scripts, because e.g. "split horizontal" and "split vertical" are implemented by opening a container and starting a new instance of the editor that will fill that area on screen and attach to the same view (the buffers are all held in a server, similar to if you use Emacs in client server mode).
It also contains no code for picking files, because it spawns an external one-liner which uses rofi to select files (same for themes etc.).
After a while, I started thinking about minor nuisances as a result of my terminal. E.g. I like a high degree of translucency, but that makes text hard to read unless there's an outline, so I started hacking outlines into st and kitty, but found other things that annoyed me, and before you knew it I had a "I bet I could replace this" moment.
I don't really care about packing up these for others as-is because so much is specific to how I want things, but I am slowly working my way through and pulling out generic components to release, and then I'll clean up the pase so it's at least there to look at.
EDIT: Just to add that e.g. for the terminal, the entire codebase, including packaging up all dependences other than the standard Ruby library, is currently ~6.5k lines. That includes ca. 1.5k lines for the Toml parser for the config files, for example, so I think it can be trimmed down significantly. For comparison `st` is around 8k lines, excluding dependencies like Xlib, freetype etc, and Xterm is around 88k lines. That's not to crow about how tiny mine is, because a lot of it I get "for free" thanks to Ruby, and a lot of it is down to just not implementing features I don't care about. But it is an illustration of how you can start really small and make something usable when you're free to keep a really tight focus. "Just" the terminal codebase itself is about 1.5k lines. But it will fail to correctly render a bunch of apps that I don't care about because there are still escapes I haven't bothered to implement. That's fine for me, but not for a terminal for general use.
I definitely love that feeling of building just for myself. In the past year I've adopted emacs in hopes to achieve the sort of agency you have to define the experience you're having on your computer.
So I completely understand the motivation, it's just that terminal and text editor seem like daunting projects. I'm SO so busy at work, and after work I need to unplug. I can justify carving out 10-15% of my time to sharpen my axe, but i just assume that a text editor took a team of people years to create.
I think you are highly productive developer.
Cheers
Thanks. But keep in mind to me programming has been a hobby since childhood. It is what I do to unplug from work - just different types of projects. That makes all the difference in terms of motivation, and energy.
That said, while an editor like Emacs is a big project, an editor can start really small. E.g. look up Femto on GitHub, which was my starting point, or Antirez' "Kilo", so named because its about 1k lines.
If you ever get the itch, look at existing small editors and pick one to butcher as a starting point. You can replace every single line over time, but starting from something working let's you focus on what you want it to be rather than how to get it to a point where you can start using it (and even if not something you can use for everything, at least now and again). It's a lot easier to stay with a project like this is you get to where you use it every day, and every improvement makes things a bit easier.
Sometimes I think a shitty visible working prototype is a better goal than a more elaborate deep prototype that takes more time to see results.
for example, I know you can design in a log system, but you can start with printf and do the logging later. It's more important to tickle your motivation brain cells with something that works.
Or, you can start with a working project, strip stuff out, and get something visible. Then add your code.
There's also another (related?) trick - when something is working stop in the middle of a feature that is almost done. The next day you tickle those motivation braincells when you can dive right in and finish it instead of diving into the motivation-killing "planning the next step" stage.
Thanks for the kind words :) It's currently in a bit of a hiatus. While the games "engine" is in a workable state it lacks content. Unfortunately I like building systems far more than actually creating game content. Might need to find a few people that are interested in helping with that at some point
Well lucky for you I like building content but lack experience in building engines (something I’d love to learn). Happy to contribute when I have some time. Email is in my profile if you feel like reaching out.
What a coincidence! In case you are also using discord there is a lonely server linked in the README. Feel free to hop in and don't worry about time. There are no strings attached. I'm in no rush and this is a fun-first project :)
Mitchell Hashimoto, Hashicorps longest serving IC, has been working on his own terminal emulator as a side project: https://mitchellh.com/ghostty . It's been interesting to read through his logs and see how it develops along with the gnarly bugs he gets to work through.
That's just my bad attempt at a joke. "Longest serving code writer" may be more accurate, but IC gets lumped in as that so frequently and he publicly took that title back on. This is getting less funny the more I try to explain it..
I wanted to write a terminal emulator, the biggest hurdle is understanding the escape sequences. all documents seem to be unreadable, including those mentioned in the post, and those often referenced in projects, like https://vt100.net/emu/dec_ansi_parser
the second difficulty is handling reflow. a real terminal can't resize its screen, but an emulator can. how to implement that correctly with cursor movements?
the third difficulty is handling font fallback and rendering emojis and other combinatory glyphs correctly.
I've written ANSI terminal handling before. The original VT100 paper manual that came with the terminal is a nice start, because the complexity hadn't really happened yet and people used to write useful documentation. With that as a starting point for understanding it isn't hard to extend into handling the full spec.
That diagram you linked is actually quite nice, but visually intimidating. If you think of it as a handful of regular expressions exploded into a state diagram that helps. For instance, the entire left half of the diagram is just the CSI code acceptor, see "CSI (Control Sequence Introducer) sequences" on the "ANSI escape code" wikipedia page.
You can write a regular expression to match a CSI and carry on. This is 2023 and you aren't using an 8080 with 3k of RAM. (probably) The only tiny trick is that you have to handle the "incomplete trailing regex" and wait for more data to arrive and try again.
As for handling reflow. I wouldn't call that an "implementation" problem. I'd call it a "specification" problem. I'd approach it by seeing what Apple's Terminal program does, write that down, and call that my specification.
You can get something going with just the most rudimentary escape handling just by spitting what programs write to your terminal to debug output in the terminal you run your new terminal from, and add a proper parser a bit later.
You can totally ignore reflow. It'll look ugly. It doesn't matter. When running full screen applications you need to handle width/height reporting, that's all.
Font fallback and nice font handling is a detail to worry about well down the line. There are libraries that can do a lot of the lifting for you depending on language/platform. Just pick a font with reasonable coverage and worry about the rest later.
> the second difficulty is handling reflow. a real terminal can't resize its screen, but an emulator can. how to implement that correctly with cursor movements?
The answer to this one, incidentally, is ¯\_(ツ)_/¯.
Every terminal emulator engine I've examined handles reflow a little differently. There's no official spec, and everyone has their own idea about how to make it work "best". That being said, I've wrestled with these questions myself, and here's what I've come up with:
* Each line should have a flag on it which tracks whether it should be reflowed with the next line. By default, this flag should not be set.
* The reflow flag should only be set when the cursor leaves that line because a character was printed with the cursor in the last column ("wrapnext" state), and no special modes are set which change the wrapping columns.
* Performing any other action which changes the contents of a line should clear the reflow flag on that line.
The same logic can also be used to join wrapped lines for copy/paste. It won't work 100% of the time, but it will work for most situations.
I always used Wikipedia's article on ANSI escape sequences. A few details could be explained a bit better but overall I found it useful. The diagram you linked is probably a more complete and compact overview of all possible combinations, but I don't find it very intuitive either.
I think it lacks some important escape sequences. E.g. how do programs like vim and tmux switch to another buffer and then restore the buffer? I vaguely know about it but never saw an actually complete documentation.
The diagram is complete. It shows the collection of the raw sequences, which includes a bunch of parameters that you then need to process separately to determine what to actually do.
To switch to/from the alternate screen mode is \e[?47h and \e[?47l. "\e[?" is DEC private mode which are DEC private mode codes. The number specifies a range of settings to switch on or off. The "h" and "l" determines if you're setting or clearing the setting respectively.
The parsing of those are handled by the escape, csi entry, and csi param boxes in the diagram.
They need to keep the screen and scrollback history in memory, then redraw it as needed. As mentioned next door, xterm has support for alternate screen mode which can save and restore the current contents. But tmux works fine with more than 2 buffers and tmux works fine if the terminal underneath it doesn't support alternate screen mode (like a physical vt100)
Ignore all that ANSI stuff, implement escape sequences like you think its easy to implement and then write a terminfo file for that so applications know how to use it.
> the second difficulty is handling reflow. a real terminal can't resize its screen, but an emulator can. how to implement that correctly with cursor movements?
That's not entirely true. Most serial terminals did have multiple text size modes eg 80x25 but also 128x50 or something. I still have a real VT520 here that can do that. Not a Dec labeled one sadly but one of the later boundless models.
I can confirm that writing a terminal is fun, for the reasons mentioned in the article: it’s easy to get “self-hosting”, but then the possibilities are endless :)
In my case, this was about creating the terminal for EndBASIC (https://www.endbasic.dev/). I wanted to mix text and graphics in the same console, so I had to ditch Xterm.js and create my own thing. It was really exciting to see graphics rendering mix with text “just fine” when I was able to render the first line.
yeso makes it pretty easy to get pixels on the screen (opposite extreme from 'OpenGL code is very heavy on boiler plate and repetition'), but its text handling capabilities are limited at best
> It’s also possible to write a terminal in a terminal, like tmux, but I’d save this for my second attempt. It’s very helpful to have a place to dump logging info that’s not also the screen we’re writing to.
I don't fully understand what the author is saying here or precisely what they mean by writing a terminal in a terminal. From my perspective though, it is easier to write a hosted terminal that runs inside of an existing terminal. Writing the full thing from scratch is a much harder problem. A terminal has many subproblems that are best attacked separately in my opinion.
At its heart, a terminal reads formatted text from standard input and writes formatted text to standard output. It is essentially a REPL. So the first step is to write a (R)ead function. Then you pass the result of read to the (E)valuate function which will process the input and finally pass it in to the (P)rint function. If you start with a hosted terminal, the read and print functions can be modeled with posix read and write so you can devote most of your time to the evaluate function.
Once you have a good evaluate function and the terminal works as you like in the hosted environment, then it makes sense to go back and write new implementations of read and write that target a new host environment. This is when it makes sense to switch to QT or opengl: when you have already implemented the core logic of the terminal and want better io performance. But it also might make sense to target html/js for maximum portability. You can either reuse or rewrite the backend that was used in the bootstrap terminal depending on how it was written. Even if you are changing languages, the rewrite should be much easier than the initial implementation since you already know what functionality is necessary and how to do it.
If you start with QT or opengl, you might never even get to a useful terminal because you get so bogged down in the incidental details.
What I am describing is essentially quite similar to bootstrapping a new programming language. The initial implementation should be done in the most convenient language/environment possible for the author. People commonly make the mistake of implementing a bootstrap compiler in a low level language, which is almost always a premature optimization and forces you to take on accidental complexity (such as memory management) that is secondary to your primary goals. Remember Fred Brook's advice to plan to throw the first implementation away. It is so much easier to do something that you have already done before than something new.
The entire backend rendering to raw X11 calls for my personal terminals is ~160 lines of Ruby, and that includes support for oddities like double width/double height, and optimizations you can drop at first like scroll up/down (as opposed to taking the slow approach of redrawing, which is enough for a first approximation). You need very little to do the bare minimum graphical output.
No current version, but I'm preparing it. But actually, to see a really ridiculously minimalist start, this was my starting point, which used a tiny C extension to do the X rendering (though it optimistically included a dummy class intended to be the start for the Ruby X backend). It's awfully limited, and awfully broken, but it shows how little it takes to be able to start writing:
It's totally useless for anything other that testing or expanding on, but it was the starting point for the terminal I now run every day, and I'll be updating that repo as I clean up my current version at some point.
The current version uses this for a pure Ruby (no Xlib) X11 client implementation:
And this pure-Ruby TrueType font renderer (I did the Ruby conversion; the C code it's based on was not mine, and is a beautiful example of compact C - look up libschrift) as I got tired of using the bitmap fonts and didn't want to add a FreeType dependency (the renderer is ~500 lines of code):
i look forward to seeing it! the simple, approachable truetype implementation is very exciting already! i didn't know about libschrift, but one third the code makes Skrift three times as approachable
Also to add that one thing that really inspired me with libschrift is the realization that Trutetype is conceptually really simple. Most of the code is parsing the annoying format. The actual rendering - as long as you don't deal with hinting, is just a matter of rasterising lines and quadratic bezier curves.
Handly OpenType then adds cubic bezier.
If, on the other hand, you want colour emojis, you need to implement a subset of SVG (though the subset is small).. Yikes.
You get 90% for 10% of the effort... As usual, I guess. Part of me want to see how far down full emoji font support can be golfed, but another part of me feels that's downright masochism.
i guess hinting is less important today than it was in the days of monochrome 1152×900 21" monitors and sub-mebibyte framebuggers where you couldn't do antialiasing
(and maybe you can do a decent job of blind "hinting" with smarter dsp algorithms, gradient descent, and orders of magnitude more cpu, and just ignore the tt hints virtual machine)
well-defined subsets of svg are a pretty interesting thing; i feel like a smol computing system might benefit from using something like that as its base graphical layer for things like fonts and windows; then a font engine can delegate the rasterizing down to the svg layer
parsing annoying formats is what https://github.com/abiggerhammer/hammer is for; we were able to get it to parse pdf. unfortunately it's not pure ruby. the smallest i've been able to get a peg parser library is http://canonical.org/~kragen/sw/dev3/tack.py which is 27 lines of python, but i'm not sure it can be reasonably extended to do the kind of binary parsing that hammer does without balooning in size
Similar thoughts on hinting - it's "good enough" for me on full HD w/just the antialiasing. With respect to automatic hinting, as I understand it Freetype has autohinting based on heuristics as hints in the fonts are often poor, but I haven't looked at how complex the auto hinter is.
Thanks for the nile and gezira link - it'd be fun to handle the emoji fonts too, at least as an addon...
Hammer looks interesting. I'm right in the middle of yet another parser combinator library in Ruby, but focused on text (it annoyed me that the Ruby Toml parser pulled in a 1500 line dependency for several hundreds of lines of excessively verbose parser definition for a grammar that can be defined in ca 50 lines) and frankly more an excuse to toy with refinements to see if I can get closer to BNF for specifying it.
I might have a look at hammer for inspiration for ways to shrink the ttf parser later.
yeah, hammer is maybe small for a c parsing library but in absolute terms isn't that small (about 10 kloc if you leave out the bindings) but it might be good as a reference point for api design
because i'm not a fan of excessively verbose things, i've been tossing around ideas for a new non-parser-combinator-based peg parsing system called 'the monkey's paw' whose grammars look like this
which you can make even more bnf-like if you want by putting <digits> and <var> on their own lines
based on my experience with peg-bootstrap i suspect i can probably implement this metacircularly in about 100 lines of code in something like lua, js, or ruby, but getting it built in a non-self-referential way will take another 100 or so. i wasn't planning on doing hammer-style bitfields and stuff, and though i might end up with hammer-style parser combinators for bootstrapping, the idea is to use a host-language-independent grammar syntax as the main way to define grammars
> that's not really closer to bnf so maybe it's not what you're looking for
I'm really toying with a variety of things. One thing is the smaller/cleaner parsing of binary formats.
The other thing I'm playing with is seeing how close I get to cleanly expressing the grammars in pure Ruby, without parsing an external format. The two things are pretty orthogonal, and always enjoy looking at new ideas for parsing in general, not just what I can abuse the Ruby parser to handle directly...
Your format is interesting. I read it as the <name: ...> bit serving as a capture? If were to map that to Ruby, I'd probably use the Hash syntax to try to approximate it, so you'd end up with something "{expr: {first: ...}". Incorporating the actions without some extra clutter would be a bit tricky, because the operator overload's won't let you add a block, and so you're stuck with a minimum of ->() {...} or lambda { ... }, but I think representing it reasonably cleanly would be possible (of course this looks like a simple enough format to just parse directly as well).
For a taste of how close to avoiding "host language noise" you can get when embedding the grammars directly in Ruby (I've not settled on the specific operators, but it's a bit limited since you can't change the precedence, I do think the '/' which came from ABNF might have been a poor choice and I might revert to '|' in particular), here's a small fragment of the Toml grammar that will parse as valid Ruby with the right code:
But that so far excludes captures. I'm tempted to default to simply capturing the terms by name.
The trick to making the above valid Ruby that this is defined inside a class that "uses" a set of refinements[1] that lexically overrides methods on a number of core classes only within the grammar definition, and then it instance_eval's the grammar within an object where method_missing returns an object that acts as a reference to named rules, so each rule is basically just a series of chained method calls (hence the annoying '&' - I think it may be viable to get rid of it by keeping track of state in the object whose method_missing gets invoked, but I'm not sure if it'll be robust enough to be worth the slight reduction in visual clutter)
Without the refinement feature we'd have to wrap the integers etc. if we didn't want to monkey-patch the standard classes and break everything. (This kind of DSL is the only really useful case I've found for the refinements feature so far)
I really should just reproduce that output in its entirety for direct comparison.
If it does not produce pixel for pixel identical output it's a bug. But here is the first pass of code that uses the same approach to integrate it with X and use Xrender to output the rendered glyphs:
(EDIT: Fixed the issue below)
Note that the example contains a hard coded visual. I need to push a fix for that - it was last updated before I added support to the X11 client to look up the visual. In it's current form it probably won't work for you. Will see if I can push a fix for that today.
nice! probably magenta on a red background is undesirable, though; maybe switch to colors with better contrasts, or add a border or drop shadow, or just postprocess with a zero-phase high-pass filter
what do you think an idiomatic ruby binding for yeso would look like? so far i only have lua, c, and python bindings, which are described in https://gitlab.com/kragen/bubbleos/blob/master/yeso/README.m.... my ruby experience is both limited (only a few thousand lines of code) and over a decade ago
Yeah, the colour choices were pretty random, I need to clean it up.
X11 visuals are severely annoying. I get they made sense in the 80's and 90's - I spent time in computer labs where the machines ranged from monochrome to SGI Indy's, with 8-bit palette, and 15 and 16 bit direct color displays in between, but I'm all for just assuming 32-bit bgra is available these days.
Any server supporting XRender needs to support it anyway as it's one of the standard formats for XRender. (I don't know if you've looked at the XRender library code, though, but one of the things that made me facepalm was that even though it requires a handful of standard visuals, the new Xrender call that returns the visuals for Xrender does not just return a list of the handful of standard formats as part of the response - you need to filter the full list of visuals and match them to depths and bit masks... Not that it's a lot of code, but it just so aggravatingly pointless)
> what do you think an idiomatic ruby binding for yeso would look like?
I think a bit of a mix between the Python and Lua would get you pretty close. Don't look at "my" X11 client for idiomatic Ruby :) It was "inherited" from an older client, and it's very closely following the protocol more so than idiomatic Ruby, and I'll eventually try to massage it into something nicer.
To take your examples, we'd probably want to use blocks. E.g. I could see the munching squares example looking something like this in
Ruby:
def munch(t, fb)
fb.each_row do |y|
fb.each_col do |x|
fb[x,y] = ...
end
end
end
You could also use the approach you use more directly, and I'm sure nobody would think that was weird either:
def much (t,fb)
fb.each_row do |y, p|
fb.each_col do |x|
p[x] = ...
end
end
end
For the main bit, "new" will always directly return the window,
but it's not unreasonable to then either `yield self if block_given?` in the constructor to allow similar code to your Python version:
Yeso::Window.new(...) do |w| .. and so on; end
or wrap that in a class method:
Yeso::Window.create(...) do |w|
(0..).each {|t| w.frame {|fb| much(t,fb) } }
end
(Endless infinite range is "new" since 2.6)
The class method vs. constructor only really buys you that you can make it slightly harder to accidentally keep the object around since you can prevent it from being directly returned. I think overall that's probably the most common approach even when the constructor
supports taking a block directly.
Why would I want to? Terminal emulators are one of the categories of software with the most options available already.
And understanding terminal codes which are historically a bunch of kludges on kludges is a guarantee for bugs. I don't think there's any terminal emulator that does it perfectly and the quirks of the most prolific ones like xterm have simply become the ad-hoc standard.
I built a fake terminal on my website[0]. I've been planning on building an actual one that is compiled to WASM, but it was fun building the little features such as a memory of entered commands that can be navigated by pressing up and down with the arrow keys. This looks like a great resource for me to take it to the next level. Are there any concerns I should be aware of if I were to deploy a working terminal on a website?
- First step (for a UTF-8-input terminal) is interpreting the input bytestream as UTF-8 and "lexing" into a stream of Unicode Scalar Values (https://www.unicode.org/versions/Unicode15.1.0/ch03.pdf#P.12... ; https://github.com/mobile-shell/mosh/blob/master/src/termina...).
- Second step is "parsing" the scalar values by running them through the DEC parser/state machine. This is independent of the escape sequences (https://vt100.net/emu/dec_ansi_parser ; https://github.com/mobile-shell/mosh/blob/master/src/termina...).
- And then the third step is for the terminal to execute the dispatch/execute/etc. actions coming from the parser, which is where the escape sequences and control chars get implemented (https://www.vt100.net/docs/vt220-rm/ ; https://invisible-island.net/xterm/ctlseqs/ctlseqs.html ; https://github.com/mobile-shell/mosh/blob/master/src/termina...).
Without this separation, it's easier to end up with bugs where, e.g., a UTF-8 sequence or an ANSI escape sequence is treated differently if it's split between read() calls (https://bugs.chromium.org/p/chromium/issues/detail?id=212702), or invalid input isn't correctly recovered-from, etc.