> Ramey told me via email that Readline, far from being an original idea, was created to implement functionality prescribed by the POSIX specification, which in the late 1980s had just been created.
This actually isn't true. The POSIX shell spec (1003 IIRC) was published in 1992. The headline library was in the first bash release; Brian might have written it before starting on bash. The inspiration was the TOPS-20 input JSYS (system call) mixed in with Emacs.
How do I know? Well you can ask Brian; as he reminded me it came from design discussions for bash I had with him on this very topic at the time, specifically that interactive text input should be a library.
And I trust Gumby's memory almost to a fault :-) I had already written a version of Emacs for the Apple ][ and Apple //e called Amacs in 1983 or so. I loved Emacs and Zmacs, and wanted to build software that had that clarity of architecture. So I knew I wanted readline to have as much of the feature set of a genuine editor as made sense.
I think Gumby was instrumental in talking to me about making a reasonable API for the completion part of readline. But I wrote a lot of code in the late 80s. And I wanted to use command line editing and completion in GDB, because we didn't have it. And I thought that was idiotic.
Hi, any thoughts on what would be a good place to start if someone was looking to write a terminal shell variant these days? Also, many thanks for all the useful things. :)
I knew I was going to write readline, and I knew I was going to write it as a library. I wrote the vast majority of it on a weekend after I had a working shell. POSIX was mostly me telling David Korn that just because he had created a shitty user interface feature, we shouldn't be required to make that part of the specification.
I trust Brian's memory on this. The original email that he and I exchanged about readline from 1989 referred to readline.c as a single source file, was it part of a separate library then?
I, unfortunately, don't have bash sources before bash-1.02.
> TENEX was created at Bolt, Beranek & Newman (a Cambridge, Massachusetts think tank) in 1972 as an experiment in demand-paged virtual memory operating systems. They built a new pager for the DEC PDP-10 and created the OS to go with it. It was extremely successful in academia.
> In 1975, DEC brought out a new model of the PDP-10, the KL10; they intended to have only a version of TENEX, which they had licensed from BBN, for the new box. They called their version TOPS-20 (their capitalization is trademarked). A lot of TOPS-10 users (`The OPerating System for PDP-10') objected; thus DEC found themselves supporting two incompatible systems on the same hardware--but then there were 6 on the PDP-11!
> TENEX, and TOPS-20 to version 3, had command completion via a user-code-level subroutine library called ULTCMD. With version 3, DEC moved all that capability and more into the monitor (`kernel' for you Unix types), accessed by the COMND% JSYS (`Jump to SYStem' instruction, the supervisor call mechanism [are my IBM roots also showing?]).
Purely as a side note, TOPS-20 was commonly called TWENEX ("Twenty TENEX", because the machines it ran on were branded DECSYSTEM-20 by DEC) and the ITS Emacs info file had an interesting note to this effect:
Twenex Twenex is the operating system which DEC likes to call
"TOPS-20". However, a person should not be forced to
call a system "tops" unless he really thinks so. Come
now, DEC, don't you think people will praise your
products voluntarily? The name "Twenex" is also more
appropriate because Twenex was developed from the Tenex
system, and has no relationship to "TOPS-10". What's
more, it's very euphonious.
The 10s at MIT ran a homegrown o/s (as had the 6 before them...there was no pdp-6 os available the first was delivered) until the OZ KL-20 was delivered. Allegedly arpa wanted security though I think that was an excuse. I believe ultcmd was written by ai lab graduate Warren Teitelman, inventor of DWIM, among other UI innovations.
Maintained by a single person, but I doubt that individual is irreplaceable as maintainer.
One reason we get so much bang-for-the-buck with software is that the maintenance costs for a given software package aren't necessarily proportional to the number of places/times it gets used.
> It's not that big, and there isn't that much that needs change.
Not... That... Big? It is a _huge_ library. A naive wc -l of just the c files, and none of the headers, lists a result of 30,654.
That's not a small project. And as it's a library and not a self-contained application, that's one of the bigger libraries around. In fact that's one of the sighted inspirations for linenoise, that readline is too big.
I do the exact same thing. This will probably be considered sacrilege by some but I also remap tab completion to cycle through all matches instead of stopping at the first ambiguous character:
# Set up tab and Shift-tab to cycle through completion options
Because instead of helping me enter a long filename by its distinctive parts, it forces flat mental read-decide-skip loop based on first few letters, which is slow and cannot be narrowed down without typing out additional letters by hand. It is like selecting from a popup menu through a 1-line window.
If it presented a real popup for file selection, combining both worlds (tab to longest match, untab to undo, up-down to navigate), that would be great. But it doesn’t.
> which is slow and cannot be narrowed down without typing out additional letters by hand.
In a PowerShell environment specifically, you usually end up having to backspace a dozen or more characters completed from a long command name before you can resume narrowing the search. It's less of a problem on a Unix shell where commands tend to be short.
Both are valid undo bindings in emacs so this is probably the reason that they are both supported. (I think this is an arrogant of a time where the keys were indistinguishable (and I think maybe they still are mostly indistinguishable to terminal apps))
And if you ever find yourself using a command-line utility that doesn't have readline support (`tclsh` and `smlnj` come to mind), you can use `rlwrap`, available via your package manager of choice:
Sigh... it always frustrated me that the Tcl folks never got something like readline integrated. IDK about the legal details, but Ruby and Python somehow managed.
tkcon is an alternative, graphical shell for Tcl that's a good alternative to using tclsh on the command line, BTW.
You're going to have much happier users if you do that thing that "anyone could do" rather than ask that each person who might want to use readline read up on Tcl's C API and hack together something that links to readline.
Also, according to RMS, CLISP is a derivative work of readline because of the optional dependency on it. (CLISP is now GNU CLISP, but was not originally part of the gnu project).
Imagine you are a photographer. You take a photo of something. The photo is covered by copyright. I can't use it without your permission.
Now imagine that I write a book and I include your photo in it. This is an infringement of copyright. I can't distribute my book, containing your photo, without infringing copyright. It doesn't matter how much your photo contributes to my book -- including it in my book and distributing it is infringement. You might even be able to get an injunction to stop me from distributing my book without removing your photo.
Now imagine that you put your photo under a CC "share alike" license. What it says is that, while I don't generally have the right to use your photo in my book, if I decide to license my book using a CC "share alike" license, then you will give me permission to use your photo. To make things simple, you give blanket permission to anyone to use the photo as long as whatever is including the photo is licensed as CC "share alike" -- in that case it is not necessary to get specific permission to use it. Finally, let's say that you make the offer that if I infringe your copyright by including the photo in my project that you will forgive that infringement if I agree to re-license the project under CC "share alike".
> All Creative Commons licenses (including the version 4.0 licenses) allow licensed material to be included in collections such as anthologies, encyclopedias, and broadcasts. You may choose a license for the collection, however this does not change the license applicable to the original material.
IANAL, so whether a book using a photo as an illustration would count as a collection is beyond me, but e.g. an encyclopedia or photo book seems to be within scope.
Now imagine I go there and take the exact same photo myself, and replace the original.
Which is exactly why license tainting like that is not a thing and never been proven to be. If I break rules, I don't lose rights to my own works, I just pay with money and paying reparations is an entirely different thing.
Yes, you are correct. Infringement is infringement. Keep in mind that you are never compelled to license your work under the GPL even if you infringe the copyright of GPL code. It is an offer to remedy the infringement. You can accept the offer or you can decide not to accept the offer. It's your choice. However, even if you replace the infringing use with a non-infringing use, you still have to deal with the original infringement. You can't really escape that.
I was on my walk and I realised that I completely missed the interesting part of this discussion (which was hinted at by the OP). So I'll do my best now (with the caveat that this is a very complex legal issue and I am not a lawyer).
One of the most crucial things about the GPL is that it is only triggered on distribution. You are specifically granted permission to use the software for any purpose (freedom 0). This includes embedding it in another piece of software. As long as you don't distribute the result, then you are 100% OK ("distribution" has a specific definition which is stated in the license, so refer to that if you have any questions about how that works).
The "anti-GPL hack" that some people use is that they say, "Well, if I don't distribute a library that I use, then I am not infringing on the license -- because the license only kicks in on distribution. If the user links the software, then it's 100% OK because they are specifically given permission to use the software for any purpose in the license. Whoo hoo!"
This is where the argument of "derived work" comes in. Imagine that you write a poem. I then write a critique of that poem. I don't include the poem in my critique because I don't have the right to distribute it. Just referencing your poem is not an infringement of copyright. Just because I describe your poem doesn't mean it's a derived work.
However, imagine that I decide to distribute my critique in a web page. Many of the things I say depend on the actual text of the poem, so I'd really like to add it to my text. But I can't distribute the poem. What to do? You are distributing your poem on the internet too, but you have a license that doesn't allow me to use it directly. Maybe I can be a bit clever here.
What I'll do is write a web page that deep-links to the poem. I'll write some javascript that allows the user to press a button and it will copy the text of the poem into my document. That way I can refer to the poem without actually distributing it. My document won't make any sense if the user doesn't press the button to link to the original text, but the user will be doing that, not me.
Is my website a derived work of the poem? This is a good question. I think it probably is. My document does not "work" without the original poem. It depends on the actual structure of the poem in non-trivial ways. However, I am sure there are also good arguments for the opposite. Given the outcome of Oracle vs Google where the judge decided that APIs are copyrightable, I feel fairly comfortable in my position, but as far as I know it has never been decided in court. Your guess is as good as mine (and better if you are a lawyer ;-) ).
But just to reiterate -- it does not matter how much of the program infringes. If any of the program infringes than it all infringes. You can't legally distribute it. So while it might seem weird that a very small part of your code might be deemed to be a derived work of some other code and therefore you can't distribute it -- that's how it works (and, to be honest, I can't see a reasonable way of proceeding if it didn't work that way).
If Lispworks writes a compatibility layer that allows SLIME to work with Lispworks, they are not causing Lispworks to infringe on the Emacs software license.
(Side note: for your hypothetical website critiquing another copyrighted work, there is a somewhat unclear [in the sense that it's facts and circumstances dependent] fair use doctrine in the US that almost surely allows genuine critique of that work to include relevant snippets [while disallowing copying the work in its entirety under guise of critiquing it]. https://en.wikipedia.org/wiki/Fair_use#U.S._fair_use_factors )
It doesn't really matter if the code as a whole worked without readline. What matters is if the code that linked to readline worked. It's that tiny bit of code that may or may not be a derived work. If it is, then you can't distribute it. No matter how tiny it is, if you can't distribute it, then you can't distribute it.
Keep in mind that agreeing to license your work under the GPL is an offer for remedying infringement. You don't have to accept. If you don't accept, then you have to deal with the consequences just like any other infringement case.
For the side not: It is true :-) However, just to be a bit playful, I should point out that fair use is not a defense in US copyright law. If your use is fair use it is still infringing. Fair use kicks in when determining damages. So your infringing use will suffer no damages because it is fair use. However, you should always be careful when relying on fair use because it is not a defense -- your infringement case can go to court and you still have to pay your lawyers!
From what I've read/studied (though I am not a lawyer), fair use is an affirmative defense against copyright infringement in the US.
I'll have to think more about the readline interface bit you raise. My gut instinct suggests that's not quite right, but my rational brain isn't able to label/ID/argue why, so your point has some merit. I'll need to think about it some (so thanks for that [seriously]!)
Maybe there's another line input library with a compatible binary interface and MIT licensed and clisp is written to interface to that library. That would seem like it might fly.
Also not a lawyer, but if you happen to run across a reference to fair use being an affirmative defense, I would definitely be interested in seeing it.
Edit: I always thought that was why "fair dealing" was considered better, but it's definitely possible I've got it wrong.
Interestingly enough, neither of these libraries are as objectively as good as readline. For example, they do not implement a true kill ring. They aren't macro programmable. I find it almost amusing that the authors were too lazy to actually reimplement all of the extremely well documented behavior in libreadline and libhistory.
Except clisp is quite useful without readline; cmucl (a contemporary lisp implementation) lacked any readline support.
Extending this, if someone made an API compatible BSD licensed readline clone, and the configure script will link with either that or the GNU readline, depending on what is in the system, would that still need to be GPL'd?
clisp itself ended up being moot because the author was happy to use the GPL and if they hadn't they could have just removed the readline support, but it is an interesting corner case of the law.
If API's are protected by copyright, and GNU takes action on such a thing, then that would be very hypocritical, giving all the proprietary API's cloned in GPL-ed software, including GNU projects.
RMS wouldn't put readline under the library license because it thought it was too useful.
He was really upset when I wrote the library license (LGPL) in the first place, though eventually the FSF embraced it, though calling it the "lesser license".
Similarly, according to Henri Cohen (personal communication), the leading number theory program GP/PARI was licensed under the GPL because of the dependency on Readline. RMS personally requested that Cohen change the license of GP/PARI to the GPL when visiting France long ago. I licensed SageMath under the GPL initially, since it heavily depended on GP/PARI, so I didn't have a choice. That said, the GPL makes a lot of sense for software such as Pari and SageMath.
I believe they could easily have wriggled out of it.
You can't infringe on the GNU license of something that is 1) not shipped with your program and 2) your program runs fine without.
CLISP has a very good FFI which could be used to attach to a user's readline installation dynamically, if so configured by the end user.
This be packaged up in .lisp file that is not even officially bundled with CLISP. Thus if there was any shred of a basis for infringment, the only offender would be that file, which anyone could share over obvious channels. (Good luck suing a file circulating on the Internet, or having all copies of it taken down ...)
If I have a proprietary or otherwise GNU-incompatible program, and some GNU library, and as user (not redistributing anything!) want to put that together with the help of some config file on the Internet, what is the basis for stopping me? And would RMS even support a state with the level of surveillance required to catch people in such acts ...
It's more of a gray area because the same author wrote them and clearly intended them to be used together. Intent can be a very large factor in civil cases.
If the file which loads readline andmakes it available for use is GPL-ed, I don't see any legal, let alone moral problem.
I can write a readline.lisp file for, say, LispWorks (proprietary, closed-source Lisp) which loads GNU readline and binds to it. Just my file has to be GPL-ed (no distributing it in compiled format only, etc). It doesn't make LispWorks violate the GPL.
I can cheerfully combine that GPL-ed file with readline, and make a distribution that I can give to users of the proprietary Lisp.
Those users are receiving my readline.lisp and readline in accordance with the GPL, and can use these however they want.
If I can do that, someone at LispWorks could do that too.
For work Y to be a derivative work of X, some copyrighted elements of X must be included in Y, as is or in modified form (such as in a translation).
Y merely depending on X is not sufficient to make Y a derivative work.
Did RMS mean that if someone configured CLISP to actually build with readline the resulting binary was a derivative work of readline? If so, he was correct.
Or did he mean that the CLISP source code and build system itself is a derivative work of readline?
There was a similar situation back in the early '90s, concerning RIPEM (Riordan's Internet Privacy Enhanced Mail). RIPEM was distributed in source form. The source was public domain. To build it, though, you had to link the free-as-in-beer-but-not-as-in-speech RSAREF library from RSA Data Security. To build RIPEM you needed a big integer math library.
The RIPEM source could be built to use either libmp, the big integer library on most BSD systems then, or libgmp, the GNU big integer library that was on most Linux systems.
RMS got upset over this, claiming that RIPEM violated GPL. I seem to recall him acknowledging that RIPEM contained no copyrightable elements of libgmp, but rather arguing that because some people who downloaded the source would combine it with libgmp, distributing RIPEM induced this combination, and so counted as distributing libgmp.
Eventually, people on whatever mailing list or usenet group this was being discussed on (I don't remember which it was) got tired of trying to explain that there was nothing like this in US copyright law, and someone (I think it might have been the RIPEM author) wrote a new public domain library that provided the same interface as gmp for all the functions RIPEM needed, but was not gmp. It just provided the simple O(N^2) algorithms for everything, but you could link RIPEM with it instead of libgmp on Linux and it worked.
That made RMS happy, because after that someone downloading a building RIPEM on Linux was not implicitly going to build with gmp.
That was kind of a ridiculous solution, but also brilliant. It was ridiculous because whether or not Y is infringing X's copyright cannot be affected by any work, Z, that is created after X and Y, at least without access to Pym particles or the Time Stone. So all of us who were arguing over the legal issues were pretty much just wasting time and missing the point.
It was brilliant because whoever thought of it saw that the rest of us were missing the point, and saw that with a day or two of hacking he could address RMS' objection, regardless of whether or not that objection actually had any legal merit, and make the issue go away.
> The Readline API is much more extensive than that single function of course, and anyone using it can tweak all sorts of things about the library’s behavior. Library users can even add new functions that end users can configure via ~/.inputrc, meaning that Readline is very easy to extend. But, as far as I can tell, even Bash ultimately calls the simple readline() function to get input just as in the example above, though there is a lot of configuration beforehand.
I feel that was designed backwards. Instead of having facilities so applications can extend readline by adding their own line editing functions specific to them, it should have facilities so users can add their own extensions and make them applicable to all applications that use readline. That would be cool.
In particular, I would like to be able to extend it to add better multi-line editing support like what zsh's zle has.
In zle, I can use `o` and `O` to create lines below or above the cursor, `j` and `k` to move to the next or previous line, `>>` to further indent the current line, etc. It's basically a mini-version of a vi buffer, right in the command line.
In readline, you can insert a newline in the current prompt with Ctrl-V Ctrl-J, but to move between the lines, I have to move left or right beyond the beginning or end of the current line.
Even worse, readline's history format doesn't support having a single entry for multi-line commands, like zsh does. :(
Wonders of Readline become painfully apparent when you have to use a program that doesn't employ it despite having a line-by-line command input interface. But! By some stroke of luck and computer magic, Readline can be injected right into the program, at least in Unixes: there's an util called `with-readline` for that.
Some programs have the editing features but lack command history for some reason, or don't store it between sessions―this is also solved by the addition of Readline.
We (radare2 project) recently added more readline-like features in our own implementation called dietline. We cannot use the readline itself due to portability issues but tried to implement the most useful features of it. See the basic documentation in radare2 book[1]. And of course, any contribution[2] to improve our implementation is welcome.
Radare2 is supposed to work on the target platform as a debugger and RE tool, and platforms can be custom and quite limited. This is why it even provides a basic UNIX commands in the shell.
I have a love-hate relationship with Readline. While it is good once learnt, it's really hard to learn.
For beginners it's not obvious there is anything to learn, and even if they do want to know, you have to basically be told to Google for 'Readline'. It would be great if it could somehow be made more discoverable.
Back when it was written Emacs was pretty popular (well, it had been the default around the lab for about a decade). So users would use it without noticing. The same reason the standard text widgets of Mac OS support Emacs commands.
We need a program that logs sessions and tells us where we could have saved keystrokes.
I think something like the script command to record my session along with a list of rules to look for, so when I backspace 8 times to delete a word it suggests C-w. Or when I arrow to the end of line it suggests C-e.
I've been thinking about this for the past six months. If anyone has suggestions, let me know.
agree. it's pretty well referenced in most shell documentation (checked bash and pdksh).. but then, well, you have to be someone who reads shell documentation..
of course, if you're not someone who reads shell documentation, then, well, how would you find a better reference anyway..
Man pages are sometimes cryptic. When someone is first learning something, having all the details thrown in their face can be quite off-putting, at least for me
\C-a: beginning of line
$: self-insert
(: self-insert
\C-e: end of line
): self-insert
So if your prompt (with | for cursor) looks like this:
grep 'blah blah' |some_file
And you execute that macro, you get:
$(grep 'blah blah' some_file)|
Which is correctly quoted and I think always correctly quotes whatever is at the prompt (unless you e.g. get half way through a string and press enter so the beginning of the prompt is halfway through a string, or maybe if you have multiple lines)
Got a question, what is the standard workaround to the fact that Readline's forward search (Crtl-S) is also XOFF (effectively pauses the terminal)? Other than turning off software flow control via stty -- I'd like to keep that on.
At least the last time I looked at a jailbroken iOS device, it opened a terminal running bash-3.2 (the same version as on macOS). It's there, you just can't get to it.
Most "packaged" jailbreaks drop a shell binary into /bin, or outright install packages for things such as Bash and the GNU Core Utilities; iOS does not actually ship with these. (If it did, I would assume that the GPL would require that Apple provided source code for it here: https://opensource.apple.com/release/ios-1141.html .)
I don't buy it. First, why would a Linux-based jailbreak install a binary for bash-3.2, when that version is going on 13 years old? Second, there's nothing that requires Apple to make the GPL sources available in both MacOS and iOS. Doing it once is sufficient. I'm sticking with what I've seen and been told: the guts of MacOS are there, just hidden.
We used to have a simple line editor so we could edit a text file. Then GNU came along and gave us a wondrously complex editor so we could edit a single line...
This actually isn't true. The POSIX shell spec (1003 IIRC) was published in 1992. The headline library was in the first bash release; Brian might have written it before starting on bash. The inspiration was the TOPS-20 input JSYS (system call) mixed in with Emacs.
How do I know? Well you can ask Brian; as he reminded me it came from design discussions for bash I had with him on this very topic at the time, specifically that interactive text input should be a library.