Quote (from memory) from Rob Pike, a few years before he wrote the editor "sam": "Everybody writes a screen editor. It's easy to do and makes them feel important. Tell them to work on something useful".
This was not long after he had left Caltech, where he had been a system admin/system programmer for CITHEP (Caltech High Energy Physics), for Bell Labs. CITHEP hired a few undergraduates as the new admins and system programmers. I was one of those undergraduates.
Most of us used the line-oriented editor QED, but as programmers were wont to do in the early '80s, said that someday we were going to write ourselves awesome screen editors.
One evening, Karl Heuer and I were both boasting about how great our screen editors would be, and it somehow turned into an editor writing throw down. We took terminals at opposite sides of the room, and both started hacking away, designing as we coded.
This continued all night, mostly in silence, with the occasional boast ("I've got text search working!") and counter-boast ("I had that an hour ago--I'm doing regular expressions now!"), and frequent trips downstairs to the vending machines for soda and snacks.
Another of the undergraduate system admin/programmers, Norman Wilson, then arrived, and saw what Karl and I had been doing all night. He sent an email to Pike about it, and the quote at the start was Pike's reply.
In case anyone hasn't seen it this CppCon talk https://youtu.be/sPhpelUfu8Q talks about rrb-trees and how they're a great data structure for writing a text editor that can handle unlimited sized documents without locking up, and allows for editing while saving and opening, and an easy way to support an undo command.
Most text editors today do not support such functionality. With lockless data structures like an rrb-tree it becomes easy. It really is an ideal way to do write a text editor.
Are there any good ones that exist? I'm a security analyst and I frequently work with large text files, 200MB-1GB and the text editors I've tried are incredibly slow.
A lot of the time I can use grep and other command-line tools to find what I need, but given the nature of the job I don't always know what I'm looking for so having the raw file open in a text editor is sometimes the only way.
This is a text editor written in K, an APL-family language. Determining how it works is left as an exercise to the reader. I wonder whether an explanation would be longer or shorter than the tutorial in the article...
Unfortunately I believe this is written in k5, for which there is no public implementation, hence it's not possible to run this editor. (I'd be delighted to be wrong here, would be very interested to see this running)
APL is an extraordinarily concise programming language, even without taking its environment into account. This is a well known implementation of Conway's Game of Life in APL:
Thanks for that write up. Is there a decent reference or tutorial for ‘k’ online that you would recommend? I would like to learn more about it’s semantics. For example, the views concept and how old versions of a view/var? Are available.
Not to my knowledge. There are a fair number of documents[1] describing the language to be sure, but it is difficult to divorce K from its implementation(s), so k2, k3, k4, k5 (etc) are all different languages. Your best bet is to download some implementations and try it out. k4 is built-in to kx's q which has a free download[3], just enter a line with a backslash on it to get into k.
Views are just a convenient syntax for memoization. Old versions of the value aren't available.
Yes of course. If you balanced the parentheses it would be syntactically valid k. It assumes a bunch of names, and would be unlikely to do anything useful, but it would be legal.
The k7/shakti tutorial playfully notes that running one's finger along the shifted top row of the keyboard is syntactically valid.
Ha! I have a very similar experience and I agree with all your points. I really enjoy the basic experience and `kak` is my main text editor ATM, but some details and parts of the philosophy are infuriating me.
I started `breeze` as a prototype way to show kakoune author how kak could potentially work https://github.com/mawww/kakoune/issues/2590 , and later decided that actually I could maybe push it further into a full new text editor.
I'd like to make it more natural to use, and get rid of the (IMO) pointless idiosyncrasies like the plugin API being keystroke based. I'd like to also make it easy to embed into other software, so that there's some hope that it could be easily integrated into existing software, so I'm not doomed to forever switch between kakoune and classic-vi editing.
The lack of built-in window management is a major feature of kakoune for me. I try to delegate all window management to my window manager, including such things as browser tabs, so I like that kakoune is designed to be used like that. The biggest downside of kakoune's approach for me is that it makes vimdiff-functionality pretty hard to achieve.
Cool! I wanted to do similar things, neat idea. These days i'm wanting to try something similar but with a nice UI - i'm so tired of the Terminal, but it's hard to get away because editors like Kakoune and Vim are just so powerful.
My wish is that you or I (if i ever have the time) will implement something like this onto XiEditor. Ie, implement this as a feature to a lower level editor backend, so that you get the frontends "for free".
It drives me nuts that there's so much work done for the frontend side of things, but the backend of editors are being reinvented repeatedly for little gain.
I hope Xi can make a performant, hackable backend to plug into solid frontends.
I doubt this will ever happen. It's all about complexity, and in case of text editors there are layers and layers of abstractions that often (for UX reasons) have to be integrated tightly. Trying to unpack all pieces into fully generic bring-your-own implementation is going to be a lot of work upfront and in maintenance, and complexity explosion.
I really like some technical aspects of Xi, but I view it as a typical wishlist project/technical showcase. In the meantime projects like kakoune, with much smaller scale but better focus, in shorter time, build ecosystems that can be a practical Vim alternative.
Thanks, I had not known about Kakoune (or breeze). It's interesting looking at Kakoune's issue list. The idea for the editor is good, but you can see the huge of amount of work needed to make a feature complete bug free editor.
Also it's interesting to see the code when you write with the latest iteration of C++ instead of ancient versions of C.
I didn't notice actual design ideas (apart from the philosophy which sounded good to me). Do you have things on your mind on what kind of actions user can do?
The list of what already works seems just the same as in Vim.
I'm just curious what kind of modal system it actually is.
Afterwards you can follow up with building an workstation OS on a systems programming language with GC, all the way from the boot sector loading code to the graphical UI, by reading and implementing "Project Oberon".
The 2013 edition uses an FPGA instead of the original Ceres hardware.
I haven't got a copy, but as far as I know Andrew Appel's book is structured in that manner, i.e. basic steps first then more advanced topics like garbage collection later.
"Engineering a compiler" is not step by step but very readable and contained if you want to skip the discussion of (say) various scanner implementations.
There is a decent sized gap between any real compiler and a related toy or book implementation (In general), so take a look at a compiler for a language you know (that isn't C++).
I haven't found any better resource than nand2tetris, see Projects 6 to 12 where you start with an assembler and end up with a compiler [1]. There is the accompanying books [2] and coursera courses [3]. Hope this helps, best!
I haven't had the stamina to go through one but have watched some sizable chunks and it's pretty cool, even with me not really knowing much about low-level stuff like that.
Shameless self-plug, but I specificities wrote interpreterbook.com and compilerbook.com because I’m also a huge fan of start from scratch and build it line-by-line tutorials and couldn’t find such a thing for interpreters/compilers that wasn’t about a toy language. You might like it too.
I've been considering making a video tutorial on how to make a compiler from scratch. I'll try to keep it very concise yet include the very basics for anyone who isn't a computer scientist. (I am one by education).
Would you be willing to pay for it (how much)? And what would be a good platform for this?
I'm doing this to pay bills while I work on something of my own. I will use C myself (might include a couple of 'parallel' videos for Rust too wherever applicable) but the way I explain, one would be able to use any programming language.
The contents would roughtly go like this-
1) Explaining the 'theory of computation', theoretical and not really necessary for learning how to build compilers but I love this topic and it does give you formal insight in the 'power' of computers and if you're going to build non-toy compilers, may help you write more efficient algorithms.
2) Tokenization and parsing.
3) Type system
4) Symbol table
5) Assembly
6) Brief introduction to some basic optimization techniques.
I want to hear if people here have ideas on how to go about doing this. Is there a good platform for interactive online coding+slides class?
"written in C" is not compatible with "basics for anyone who isn't a computer scientist", IMO. It just gets you so bogged down with low level details which is not necessary for an introductory course.
I still haven't found a better introductory text than Appel's "Modern Compiler Implementation in ML". I also like pairing this with "Language Implementation Patterns", which uses ANTLR.
Video makes it easy to ramble about design-choices, etc. But I'd rather read content. Seeing snippets of text on-screen, at my own pace, is much better than seeing shots of a monitor or slides. Much like this story.
That said there are a couple of great books out there already, interpreterbook.com, compilerbook.com, and craftinginterpreters.com - all recommended. If you're doin g this for money you'll struggle, unless you have a lot of clarity / something new to offer.
There is an old one on compilers called "Let's build a compiler by Jack Crenshaw" Its in Pascal but I think there are modern rewrites of it (like C on an x86)
Compilers have a lot of relatively independent pieces, unlike a text editor. Better luck at looking at those pieces instead (Lexing, parsing, type checking, etc...)
For your information, even a simple a text editor (like much of user-facing software) too is comprised of several logically independent pieces, in the spirit of the model-view-controller paradigm.
A long time ago I had to write a Windows custom control for text rendering. This was really hard to do in a performant way once you dealt with different fonts, pages sizes, scrolling and all the other stuff we usually take for granted. One of the most painful months in my life but I learned a lot since nobody told me how to do this so I had to make it all up.
Edit: oops, yours is in assembly, so definitely not in the same vein :-p (except for both trying to be small!)
Edit2: ... and, the article is about kilo. D'oh. This is what happens when you go first through HN comments, only to later open the article (which I do often!).
My big fear about implementing a text editor is writing a rope data structure to be able to edit large files. I don't really know if it's mandatory for all text editors though.
I was a little frustrated with sublime text folding code using indentation instead of syntax (there's an issue but they don't want to fix it). I have large C++ files, and it seems visual studio does a better jobs at folding.
It's not mandatory. There should be good rope implementations already, though. If you're in Rust there's ropey and at least one other pretty good one.
On modern computers, you can get away with a flat buffer of text (not even a gap buffer) for documents up to a few megabytes. I don't really recommend a gap, because it adds complexity and doesn't help with the worst case, though of course it cuts your average case down.
If you're going for simpler than a rope, my recommendation is array of lines. You have to do logic to split and fuse lines (for example, when backspacing over a newline), but it's not too bad. The only thing they don't do really well is single long lines.
I don't recommend piece tables. They have superb performance on first load, but then fragment. The reason I'm such a huge fan of ropes is that they perform excellently in the worst case - long edit sessions, long lines.
In my text editor, I just use a doubly linked list of lines, instead of a rope. It doesn't limit the performance in any situation I've found. A performance problem I do have is large column mode edits. eg if I want to delete the first columns of the entire file (typically some log file with timestamps at the start). I suspect the rope data structure would make that worse.
I like the simplicity of the doubly linked list of lines. I'm not sure what advantage the rope data structure would bring.
Doubly linked list of lines tends to be slow on, for example, editing documents with very long "lines" e.g. doing a find/replace on a 10mb json that's not pretty-printed and is a single line.
If you use c++ you can start with a std::vector. Computers are really fast. Don’t optimize prematurely. And hey, maybe you don’t need a rope. Just put std::vectors to a std::map or std::list per row of text?
Lots of problems that have an optimal complex solution can be solved with way simpler constructs in the prototyping phaze.
I'd go further than that. I'd say in production too. If the simple thing is fast enough even when you test on a slow machine, then implementing the faster but more complicated data structure is a form of premature optimization.
As another commenter points out, using RRB trees is a great option! The immer version is both fast and easy to work with and you get undo for free (and even undo trees).
As the author/maintainer of a once popular text editor I feel it is easy to get started then very hard to finish. Modern text editors require a lot of details and even more demands from those who use it. It’s a fun programming challenge. I still find myself thinking about awesome features to add.
How about ray tracing? Peter Shirley (whom I learned a ton from in grad school) wrote a trilogy of mini-books beginning with "Ray Tracing in One Weekend."
I don't see any references to things like HarfBuzz or DWRITE, which are NECESSARY to properly display any scripts that's not European.
All properly implemented text editors are rich text — because you will need to support font fallback and BiDi and all the Unicode complexity
You absolutely don't need any of these for a tutorial text editor... Those are all things you can hook-in later, after you've understood how to make text editors...
Did you see that this is a command line app (like VI, or EMACS)? Those libraries look like they're used for GUIs. Do you know if they can be used for the terminal?
Good stuff for reminding me that `x/sys/unix` exists and the terminal-raw-mode it allows. Should inspire someone out there eventually to plumb everything in pure Go that people would otherwise seek from ncurses, GNU readline lib etc.
if you use windows (or linux with wine) you can write self-contained apps using Autohotkey. It's free from autohotkey.com. It's usually advertised as a macro-recorder which it also does.
Disclaimer: I haven't read the article just yet. Saving it for later.
An alternative approach to this is to open vim with a blank .vimrc and add config as you need it. Nothing except what you REALLY need NOW to get things done. That's how I started my adventure with vim a few years ago and it's still paying off.
Which is what writing vim script essentially is. Or if vimscript is not enough, you can use literally any other language and integrate it via vimscript. I'm not an expert though but I have spent quite a lot of time on doing that.
I get that ViM is really powerful, so is Emacs and the like, however I'd guess a large part of going into the trouble of building your own text editor would be to learn about the algorithms and data structures used, something you can't do by writing ViM configuration files.
Well - not to be needlessly contrarian, but I've programmed that way (vim with nothing except what's included) for nearly 25 years now... it already has everything you need.
This was not long after he had left Caltech, where he had been a system admin/system programmer for CITHEP (Caltech High Energy Physics), for Bell Labs. CITHEP hired a few undergraduates as the new admins and system programmers. I was one of those undergraduates.
Most of us used the line-oriented editor QED, but as programmers were wont to do in the early '80s, said that someday we were going to write ourselves awesome screen editors.
One evening, Karl Heuer and I were both boasting about how great our screen editors would be, and it somehow turned into an editor writing throw down. We took terminals at opposite sides of the room, and both started hacking away, designing as we coded.
This continued all night, mostly in silence, with the occasional boast ("I've got text search working!") and counter-boast ("I had that an hour ago--I'm doing regular expressions now!"), and frequent trips downstairs to the vending machines for soda and snacks.
Another of the undergraduate system admin/programmers, Norman Wilson, then arrived, and saw what Karl and I had been doing all night. He sent an email to Pike about it, and the quote at the start was Pike's reply.