People who actually use Lisp (opposed to people who only write about it) have no problem that there are extensions to the standard. The whole point of Lisp is that it is easily extended in many ways: the programmable programming language. Sure it brings a bunch of problems, but that's what it is and why people use it.
The stuff he writes about CLOS is mostly wrong. CLOS is not implemented as a bunch of macros. CLOS is actually a three level architecture: the core is written object-oriented in CLOS itself, on top is a functional interface and for user convenience there is a bunch of macros for easy definition of classes, generic functions, methods and some other language objects.
Then, macros are no problem. They are a feature. No, 'hygienic' is fine, but not needed. No they don't need to be rethought. They do a useful job like they are.
No, Lisp will never be massively successful (because it is a bit too complicated, too dynamic, too powerful for many users) and bending the language trying to make it 'massively successful' won't improve anything. No, I don't want a dictator. Lisp has choice, various alternatives and a more democratic approach.
Lisp is still a good choice for computing with symbols. If you don't need that, then you might not need Lisp. But that's not a reason that Lisp should change. Should a helicopter be changed in design because someone does not use it, but drives a car? Should the helicopter be changed, because more people drive cars, getting a drivers license for cars is easier than getting a pilot's license for helicopters? Lisp is different, for a reason and this difference will ensure its survival for many years. We have now fifty years that the basic ideas have proven to be useful and I don't see that this will change in the coming years.
bending the language trying to make it 'massively successful' won't improve anything
This needs to be repeated more often - quality isn't measured by popularity. It's obvious when we talk about food establishments (McDonald's being the equivalent of Java), but for some reason, the logic is completely lost on people when we start talking about programming languages.
Is this true, or is it something you want to be true. I would say its a disservice to the language which in many ways was better than its peers and has since been imitated a lot (VMs, Javadoc, Parts of the platform library).
The tools in terms of breadth and quality available for java (IDEs, testing, decompiling, AOP ... etc.) are years ahead of most other platforms. (Excluding C#, what else...?).
A large part of this is due to the simplicity of the language (and the stability). Is java worse if you are developping in Vim, no doubt, but this is a strawman. With the tools vs. python|ruby|lisp is a different proposition entirely.
Java appealed to the industrial users: a controlled language specified down to the instruction level, no complicated features, absolute stability on all levels. The SUN Java courses were just that: the programmer has to learn the language constructs and the library. Then the programmer learns some patterns to use the language effectively
Lisp is the antithesis to this: it is a language laboratory - the programmable programming language. The programmer needs to develop skills to understand when and how to change the language. Changing and extending the language is a core mechanism. This makes the language unstable and puts a lot of linguistic power into the programmer. With Lisp you need to trust the programmer and the programmer has to learn how to justify the trust (by learning to effectively use macros, by learning when and when not to use EVAL, ...).
The Lisp programmer has to learn how to program languages and the concepts he may want to use for that.
I don't really disagree, though it is a bit of a simplification. There is a lot to this argument, but without going into details (another time maybe) is it not a testament to the base of the language that they've managed to build so much crap on top of it??.
I don't mean to say Java is all-bad. McDonald's is an extremely innovative company as well - they've put a lot of effort into optimizing the process of food production.
A large part of this is due to the simplicity of the language
I was with you until that part - maybe the abstraction Java programmers deal with, sure. But the question "what is my computer doing?" when you run a Java program is well beyond the vast majority of Java developers, partially because it's so complicated. This is a Very Bad Thing.
A better way to state this is that if you are a good cook you are willing to spend more on a quality sharp knife.
If you aren't a good cook you'll probably buy the serrated surgical steel knife from the guy in the mall because it's only 19.95 and will "stay sharp for life".
Lisp has a cost for useage. Higher learning curve, Different way of thinking about patterns. Harder to use correctly at first. It's worth it for the "good cook" to pay the cost. Maybe not for the everyone else.
(as an aside I love my expensive set of high quality knives. and always feel handicapped when I have to cook without them. But they did cost a small fortune)
I really think if you plan on programming for more than a year, the learning curve for Scheme or CL is worth it.
I was at the jQuery conference this past weekend, and a lot of the functional programming concepts would be way easier for people to understand if people saw them in Scheme first.
I learned enough Lisp in 3 months to give up C++ completely. A year later it replaced Perl for me. I can wake up in the morning and do all computing within a single Lisp instance.
I agree with him about "which implementation?" and "worthless spec." I have yet to hear a valid reason for why a modern standard library for Common Lisp is a bad thing. There are lots of reasons for not doing this, but they are all political in one way or another, not technical.
I agree with you that CLOS and macros are not problems. Common Lisp is not primarily an object oriented language, and that's OK. That it happens to have an extremely powerful object system is a nice bonus. If you want a ground up object system, use Smalltalk or Ruby and have fun.
The rant against macros was incoherent to me. Editor support for macros is imperfect so...macros suck...or something? Macros remain the distinguishing feature of Lisp, in that no other language allows you to transform your source code so seamlessly. I'm not sure there's a compelling reason to use any Lisp over other languages if macros are eliminated.
While I agree with your sentiment mostly, I feel Lisp's un-popularity probably has nothing to do with Lisp itself.
For that matter, mainstream popularity of anything has always had a good dose of randomness in it (I bet you can model it with some sort of free energy / entropy equation). For example, it's hard to break down exactly why the No.1 hit song is No.1, or that No.7 song is permanently No.7. Usually it has more to do with the politics of the record company than the quality of the song itself.
We are also being a bit pessimistic here. Lisp is not un-popular. It just hasn't reach the critical mass yet. And it seems to be going to that direction with unraveling of Clojure.
Still, I'd love to get a job using Common Lisp, which I haven't really touched since I studied. Why can't I do that?
Because, in the words of Drew Crampsie (paraphrased): Lispers do not look for Lisp jobs. They look for work to do and do it in Lisp.
Edit: Also, I believe well-established Lisp companies (such as ITA or Clozure) usually look for experienced Lispers. If you studied it in school, what makes you a good candidate for these companies?
Thanks for good information, but it wasn't what I asked about. :-)
Lots of the standard scripting jobs out there could be done at least as well with lisp, so why aren't many of them? (For instance, there has been good interpreting and compiling environments for Lisp since the 1980s? 1970s?)
(I didn't have any Common Lisp courses, I studied another lisp. I learned CL by myself.)
The argument that Lisp is too complicated for the average programmer is clearly nonsense: Anyone smart enough to use C++ to a professional standard is certainly smart enough to use Lisp. It's just sour grapes.
Try listening to Matz talk about his experience with Lisp and (subsequently) his inspiration to make Ruby. Basically, he said Lisp sounded great in theory... but actually when you have to use it it's a huge pain in the ass. And Matz is a pretty fucking smart guy, so it's not just that too many people are too stupid to use it. Lisp actually has problems. But nobody is willing to talk about them.
(I'm trying to hunt down that talk, but not having much luck.)
I saw that talk and I can tell you that he looked very confused. He was just in the need for an excuse to develop just another scripting language. You could see that he had not much experience with Lisp and that Lisp was not something that he actually needed. I think he knew Lisp mostly 'in theory'.
If you think that he is a pretty smart guy, then I would say that there are many Lisp implementations that are way ahead in terms of implementation technology - Matz's Ruby implementation is in the Lisp world several decades behind the state of the art. Ruby is one of the slowest scripting languages and has a very much ad-hoc design. It does useful things for many people, and that counts. But from the technology it is Ruby that is a huge pain in the ass. Just see how people struggle to write a decent compiler for it. Instead it is just another simple interpreter with a C core for the interpreter implementation and parts of the library. Ruby is fast where you stay in with the library functions that are implemented in C - otherwise it is a pain.
The better Lisp systems are also written in Lisp and their performance is sometimes 10 to 100 times of what Ruby offers - while Lisp offers also lots of dynamic features and the code as data paradigm.
Started with BASIC: 'I could Make Intelligence'. ???
Then he found Lisp in a magazine article and 'fell in love with it'. 'BASIC aristocracy' vs. 'Lisp democracy'???
He liked Lisp from a book, but 'unfortunately he did not have a computer that could run it'. In University he used Emacs Lisp and it did not make him 'happy'. Parentheses? No? He had no problems with it. Macros? Partially. 'Smart people just underestimate oridinarity (sic) to use Lisp'. 'We are too ordinary to use Lisp'. Aristocracy. He is happy with aristocracy, as long as he has power.
The whole point is that there's more to languages than just "technical correctness". Lisp is great technically and theoretically, but in other respects it is quite a ways behind (for instance) Ruby. So Ruby has some big problems (speed, to use everyone's favorite example) but just saying "Ruby is slower so we should all use Lisp" is a bit ridiculous. That's what he was trying to say.
(He's also not a native English speaker, so i cut him some slack on that front.)
I was not talking about 'technical correctness' (though the literature how to implement languages does exist - just start with Steele's papers - Steele describes efficient compilation some thirty years ago), but about 'technical capabilities' - and it that area Ruby is lightyears behind Lisp implementations. I'm also not saying 'Ruby is slower so we should all use Lisp', I'm saying: Ruby is much less capable than Lisp, so it neither is a Lisp nor is it able to replace Lisp for the tasks where people use Lisp. For an expert Lisp programmer there is little reason to use Ruby anyway. Ruby has a larger user community, but that does not help me when Ruby simply does not provide the capabilities I need - the user community could be ten times larger and the number of libraries could be also even larger. It does not help if the libraries that I'm interested in are not available and probably never will.
Matz designed Ruby for so-called 'ordinary' people with him to do the language design. I can tell you for sure that I neither like his design nor do I want him as the language designer. In the Lisp world the implementors role is to empower the user, not make them dependent. I need a new control structure? I write one myself. I don't have to wait for some benevolent dictator, who probably does not get basic designs right - just look at the confusion of blocks, lambdas, etc. in Ruby.
(also there is no excuse for putting confused stuff on slides, even when he is not a native speaker - lots of people (including me) are no native English speakers and I haven't sensed this amount of confusion Matz shows)
That Lisp is too complicated for some people is not news, still Emacs (which he tried) now comes with a standard library of a million lines of Emacs Lisp code for all kinds of editor extensions. Emacs Lisp code that has been written by very different people and obviously a lot of them learned enough Lisp to write sophisticated applications based on Emacs (like GNUS, calc, the various mail modes, org mode, ...).
Lisp is also not only great technically and theoretically, but some people found it practically. There weren't ten maintained Common Lisp implementations if there were no users for it. For example the talks at the Lisp Meeting last sunday here in Hamburg described several applications in Lisp: an airline reservation system (developed by a team of hundred people, fifty of them Lisp programmers), a visual reactive programming system, an aircraft analysis tool, a reasoner for the semantic web, a visual simulation system, machine learning applications and some more.
If people think popularity doesn't matter at all, they are ignoring the economics of programming languages, where it does indeed matter that others use and contribute back to the community.
Is Lisp popular enough? Hard to say, but people certainly seem to complain about the lack of popularity.
It is popular enough, that there are ten maintained Common Lisp implementations, a hundred Scheme implementations and new dialects get invented every other year.
That makes lisp a popular IDEA but it completely ignores the GP's point of economics of a language. Specifically with a dozen implementations each you end up with a bunch of libraries that are subtly (some times not so) incompatible with other implementations so while the community as a whole might be large you don't get the advantages in terms of available code that the size should bring.
At least that was my experience with Scheme, I don't know (but suspect from all the whinging) about the CL side of it.
Which is one of the big advantages of Clojure of course, it comes with a lot of batteries.
Common Lisp defines a mildly large language with a standard of 1000+ pages.
Implementations use both non-portable and portable libraries. There are also several libraries that span several implementations.
Also the implementations serve different purposes and may use widely different implementation strategies - so not all libraries might apply to a certain implementation.
The choice is there for a reason. Some libraries might start on some implementation. If it is good, then it will likely be ported (if necessary) to other implementations.
As Steve's article suggests, I don't think lots of more or less equal implementations is that great in the long term. IMO, having one main one and some experiments floating around works better.
Why do you want to take away my ability to choose a suitable implementation?
I let you have your freedom to choose a scripting language.
Lisp is a family of different languages and implementations. Like 'scripting languages': python, rebol, ruby, javascript, vbscript, ... - similar problem there: why not have one main language? why these all these scripting languages which share almost NO code?
In Common Lisp I have a core language and can share lots of code between different implementations. ECL compiles to C and byte code and can be embedded. SBCL uses an optimizing compiler to machine code. CLISP uses a byte code engine written in C. ABCL is compiles to the JVM. LispWorks and Allegro CL define very large languages with lots of bells and whistles. Each of these implementations have been developed for different usages scenarios. CLISP is fine for scripting, LispWorks is better for the development of portable applications with a GUI.
There are Common Lisp implementations that are more popular than others. For free implementations SBCL is the most popular. But there is competition. CCL is also very good. Some implementations are less popular, but they survive because they use different implementation technologies. The commercial implementations like Allegro CL and LispWorks are also widely used, and provide features that the other implementations don't provide or don't provide as complete. LispWorks for example has a cross platform GUI toolkit that is used to implement its cross platform development environment. One can develop sophisticated GUI applications that can be compiled for Windows, Unix, Linux, FreeBSD and Mac OS X without changes.
Ruby for example does not give me these choices. The main implementation is widely used, but lacks much of what the advanced Lisp implementations offer - which enables me to write a different class of applications in Lisp.
Still, I'd love to get a job using Common Lisp, which I haven't really touched since I studied. Why can't I do that?
I pay my bills writing Common Lisp code, full time. The fact that you're not an employed Lisper is your own fault, IMO, it takes guts to look someone in the eye and tell them their investment, in both time and money, is safe with you and your fringe programming language. It's allot of work, a lot of responsibility, and most certainly tons of fun.
Try it.
P.S. I am both lead developer and "CTO"; I have a team of web developers who consume my backend and architecture.
@IMO, it takes guts to look someone in the eye and tell them their investment, in both time and money, is safe with you and your fringe programming language.
Not just fringe language, but fringe anything.
The only known amongst a see of unknowns will be you.
the scripting languages are a shallow subset of Lisp, in some ways.
But Lisp was not designed to script applications. It was designed to be a language that can compute with symbols. This applied to Lisp itself lead to meta circular definitions, macros, meta level architectures like CLOS, and more. It led to the first self hosting compiler in the earlz 60s and ever since Lisp implementations have been written largely in itself.
It was designed to easily implement all kinds of notations and evaluation mechanism. Originally for example to implement symbolic mathematics and logic reasoners.
If the subset of Lisp that allows scripting is fine for you, that's great. Why complicate things?
>>If the subset of Lisp that allows scripting is fine for you, that's great.
Well, the point was that I don't see why subsets have taken over, it's as if PHP grew larger than the rest of the scripting languages for web progr... Oh, wait. :-)
But seriously, if you take the hit of a GC (which most everything do, today) then you might as well use the nicest system around. I am happy with my present tools, but a nice lisp variant might seriously tempt me.
Sigh, maybe I should just look for different work or do a startup with really hard problems.
The performance hit of a GC is less than the hit of a language that wasn't designed to be compiler-friendly. Scheme or CL will perform better than scripting languages.
You don't need really hard problems before the benefits of a language with macros become obvious. SQL-driven web apps are a lot easier to write if you have macros, for example: http://brl.codesimply.net/brl_4.html#SEC31
People who actually use Lisp (opposed to people who only write about it) have no problem that there are extensions to the standard. The whole point of Lisp is that it is easily extended in many ways: the programmable programming language. Sure it brings a bunch of problems, but that's what it is and why people use it.
The stuff he writes about CLOS is mostly wrong. CLOS is not implemented as a bunch of macros. CLOS is actually a three level architecture: the core is written object-oriented in CLOS itself, on top is a functional interface and for user convenience there is a bunch of macros for easy definition of classes, generic functions, methods and some other language objects.
Then, macros are no problem. They are a feature. No, 'hygienic' is fine, but not needed. No they don't need to be rethought. They do a useful job like they are.
No, Lisp will never be massively successful (because it is a bit too complicated, too dynamic, too powerful for many users) and bending the language trying to make it 'massively successful' won't improve anything. No, I don't want a dictator. Lisp has choice, various alternatives and a more democratic approach.
Lisp is still a good choice for computing with symbols. If you don't need that, then you might not need Lisp. But that's not a reason that Lisp should change. Should a helicopter be changed in design because someone does not use it, but drives a car? Should the helicopter be changed, because more people drive cars, getting a drivers license for cars is easier than getting a pilot's license for helicopters? Lisp is different, for a reason and this difference will ensure its survival for many years. We have now fifty years that the basic ideas have proven to be useful and I don't see that this will change in the coming years.