Lisp is a powerful language that allows one to mix multiple logical levels within a program (especially through it's homoiconicity).
Shrdlu is perhaps the most classic illustration of Lisp's power[1], A classic program from 1968-70 that allowed natural language communication about a micro world. It was written in a version of Lisp even more free-form than today. When I looked at the source code a while back, it's parsing of natural language involved a wide-variety of on-the-fly fix-ups and such to take into account the multiple irregularities of human language.
The thing about Lisp's power is it allows the production of complex program quickly but doesn't particularly have a standard way of gluing these programs together. The other classic Lisp work, The Bipolar Lisp Programmer[2], describes how Lisp has multiple partial implementations of important libraries and programs simply because it allows a certain type of personality to by themselves produce something that's remarkably good but doesn't encourage any particular group effort.
Lisp is certainly evidence that "language matters" but not evidence that Lisp is where one should stop.
> The thing about Lisp's power is it allows the production of complex program quickly but doesn't particularly have a standard way of gluing these programs together.
I'd argue Clojure is setting some new standard here with its powerful literal notation for basic data structures (list, vector, set, and maps) and its practice of integrating/composing first on data. Most of the time that's enough to get some libraries work together. If not enough, integrating/composing on functions is the next stage. Only after having covered data and functions should you consider macros, which should be easier to write if you have done a good job at the data and function levels.
If such powerful and ubiquitous data structures work so well for programs written in clojure(script), why would that not be true in the larger picture for integrating systems? JSON worked much better than XML because it actually represents simple data structures, not objects talking to each other. Redis, Avro, and similar tech are continuing that story while Kafka adds a fantastic transport and storage mechanism. Sounds like we're closer to having better foundations for integration in the small and in the large.
I have used Clojure on several customer projects and my own Cookingspace.com hobby project. I like Clojure except for the way recursion is handled. I have been using various Lisps since 1982 and Clojure’s support for recursion bugs me.
Yeah, because the JVM can't do TCO you can't have nice recursion. The options in the language provides are good enough but definitely not the most ergonomic.
I can't remember the last time I wrote a recursive call in Clojure, other than for multi-arity. In general, if I were in a code review and saw the "recur" keyword, I think I would regard it as a code smell and see if I can re-implement it using higher order functions.
> Lisp is a powerful language that allows one to mix multiple logical levels within a program (especially through it's homoiconicity).
Statement like these are what make me suspect Lisp code would be a maintenance nightmare. I am regularly refactoring code to separate multiple logical levels so we can more easily maintain our codebases. If this is a misunderstanding, can someone clarify?
>>Statement like these are what make me suspect Lisp code would be a maintenance nightmare.
After having maintained Java code bases that do 300 classes just to post a JSON to a REST end point, I'm fairly confident Lisp code can't be that hard to maintain.
There are perhaps languages out there which are better than Lisp for maintenance in some respect, but it's none of the mainstream ones. Lisp code is not particularly harder or easier to maintain; most of that depends on the original author(s) as with any codebase.
It's the reason Aaron Swartz gave for rewriting reddit in python:
>The others knew Lisp (they wrote their whole site in it) and they knew Python (they rewrote their whole site in it) and yet they decided liked Python better for this project. The Python version had less code that ran faster and was far easier to read and maintain.
I suspect the problem is that LISP is too powerful. Language power is inversely related to maintainability/readability, as per the greatly underrated rule of least power: https://en.wikipedia.org/wiki/Rule_of_least_power
> Language power is inversely related to maintainability/readability,
Is that really True?
Having worked on VB6 codebases in the past, the less powerful nature of the language just meant a LOT more code which is definitely harder to maintain.
No, it’s not, and even if it were true it wouldn’t disqualify lisp, since you can quite happily wield lisp in a context which lets you express logic in simpler/constrained forms. Consider hiccup, datalog DSLs, etc.
It's inversely proportional to length too. Maintainability is about maintaining a delicate a balancing act between several competing fundamental concerns - two of which are expressiveness and verbosity.
I think python's popularity is partly because it found a sweet spot that was neither too expressive nor too verbose. It's certainly possible to dial up the expressiveness (LISP, Perl) or dial it down (Golang, VB) and get something that is harder to maintain both ways.
Python derives a fair amount of expressiveness from the power it gives to library authors. A lot of this in turn comes from its dynamic implementation.
As a library author you can write effective python at different levels. Some involve quite heavy meta-programming (jinja2, collections.namedtuple, django's ORM) and most users would never find this out.
The law of power linked above says to use the least powerful language appropriate for the task. So it's possible VB6 fell into the category not-powerful-enough/not-appropriate.
This. I find Lisp optimal for the small band of coder heroes scenario. They must align on cs philosophy, and taste. Grow the team, or have divergent approaches: the efficacy of Lisp quickly disappears and problems appear.
I have experience of great success and productivity in small teams with Common Lisp and Clojure. I don't have direct experience of dysfunction in a larger Lisp team. That was a guess based on how being a senior dev in a 50 man C# shop felt :)
Well, at some point we have to face reality. There have been hundreds if not thousands of successful projects with teams of hundreds if not thousands of developers working on them, at the same time or over time. These projects have been generally in Cobol (ok, we can chalk that up to brute forcing a tech from they didn't know any better), C, C++, Java, C#.
There seems to be a connection between scaling a project regarding developer numbers and programming language "power". Having a smaller, shared vocabulary seems to greatly outweigh programming language "power" as the number of programmers goes up greatly.
Lisp had 60 years to prove its human scaling capabilities. So far it hasn't convinced.
> There seems to be a connection between scaling a project regarding developer numbers and programming language "power". Having a smaller, shared vocabulary seems to greatly outweigh programming language "power" as the number of programmers goes up greatly.
That goes against evidence from current enterprise software experience. The Java eco-system has the absolute largest programming vocabulary ever (J2EE, JEE, ...) and is widely used.
AT&T/Lucent once wrote the software for a telephony switch in Lisp - I think the project ran over a decade and created more than one generation of working/shipping hard&software. The team was easily 100+ people. I heard a talk of the responsible manager years ago. They wrote basically the same functionality as a ten times larger C++ project and the Lisp team lead was extremely satisfied with what they wrote.
AT&T Management favored the C++ project for mostly non-technical reasons - C++ being an 'industry language' with a larger supply of developers. Not surprising for AT&T - Ericsson made a similar decision at some point in time with their (later lifted) 'Erlang ban'.
> Lisp had 60 years to prove its human scaling capabilities. So far it hasn't convinced.
It can take more than you might expect to "convince".
> In 1601, an English sea captain did a controlled experiment to test whether lemon juice could prevent scurvy. He had four ships, three control and one experimental. The experimental group got three teaspoons of lemon juice a day while the control group received none. No one in the experimental group developed scurvy while 110 out of 278 in the control group died of scurvy. Nevertheless, citrus juice was not fully adopted to prevent scurvy until 1865.
I have heard that the British knew what caused scurvey, and how to treat it with citrus, but they kept it a closely guarded secret to keep an advantage vis-a-vis other countries navies.
That is more or less correct, but it's a very weird situation. Scurvy had been known -- along with effective cures -- for thousands of years. The method of curing scurvy hasn't been secret since -- at the latest -- the Ebers papyrus of ~1500 BC, which correctly prescribed feeding the patient an onion. And in fact cures were widely known throughout the world since that time, including in Europe. Note again that James Lancaster had heard that lemons were effective against scurvy, 200 years before the Navy got around to requiring them.
> from about 1500 to 1800 two million sailors are estimated to have died from scurvy on expeditions to Asia, Africa and the New World.
> Why no cure? In Tudor England an effective treatment, scurvy grass (a corruption of cress), was commonly recommended; at the same time the Portuguese knew a cure, and so did the Spanish and the Dutch. For unknown reasons such wisdom was applied inconsistently (even by Britain’s Royal Navy after the Napoleonic Wars) until the identification of vitamin C in the 20th century.
It looks more like a case of the people making the decisions being unfortunately disconnected from the people who knew what scurvy was and how to deal with it. (And wilfully ignoring those who tried to point it out to them.)
I know about that the whole lime - lemon thing. Two things:
1. That was before the scientific age. They didn't know about vitamin C at the time, they couldn't even see it, even if they'd somehow believe in it.
2. There is no scientific proof, 0, none, that Lisp is superior. And it's almost impossible to prove it, since it requires controlled studies at a large scale - good luck with taking away hundreds of productive programmers for that study :) All we have is hearsay and personal opinions.
Be careful; my quote has nothing to do with the confusion between lemons and limes that occurred hundreds of years later. The Navy instituted a lime juice ration in 1799. They switched out lemons ("limes") for what we would call limes in 1865, setting themselves up for the reintroduction of scurvy. But James Lancaster performed his experiment (and reported his result of 100% scurvy prevention to Naval authorities) in 1601.
I love how I'm being downvoted for saying that there's no scientific proof that Lisp is superior. Come on, show it, I want to see it. I want to see thorough, statistically representative studies that show Lisp's superiority :)
That's nowhere near something like medical clinical trials, though.
Think of studies where you:
a) have statistically representative sample size (100+ developers)
b) actual numbers and compare them (the average development time needed for the Java/C++/etc. applications was Y and the average development time needed for Lisp applications was Y - Z, where Z > 0, etc.; the average execution time for Java/C++/etc. applications, etc., you get the idea)
Computer Science studies are still in their infancy.
We have to decide what's enough to convince (and convince whom) I guess.
Just to throw out a couple of examples:
One of the larger businesses I'm aware of that's using a Lisp is Nubank in São Paulo. Valued at $1bn+ and 4 million paying customers[1]. Finance is a fairly complex domain.
King in Stockholm has also fairly recently rewritten their main game creation tooling in Clojure[2].
It strikes me as a matter of structure. Lisp provides almost none, whereas python provides more. Anything not provided by the language and environment must be provided by the team. If they can, that's great, but it's going to be easier to get a small team to agree on a structure than a large one.
Wouldn't say so, there are plenty of conventions and common practices shared by overwhelming majority of Lisp developers. Project structure, system definition, development environment, naming conventions, idiomatic use of data structures and object system, all that is established across the board.
Well I don't find mature Python codebases (e.g. Homeassistant) particularly easy to inspect or change. Guess that's where me and 2005 Aaron Swartz would disagree.
When I talked with Alexis Ohanian he told me that the Lisp version was difficult to keep running. I think the term he used was that it often fell down.
I agree, strong barriers between layers are important. Right now the best way to build them is by using different languages or runtimes - it’s hard to mix layers if they require explicit network or shell calls.
If everything was in lisp the layers are naturally blurred.
As if they've tried them and then abandoned them? Java, fone one, didn't even have generics and closures when it came out.
It didn't have macros because it was intended for the ho-hum enterprise programmer of the time, who, the thought was, would not know what to do with them.
Instead, they re-invented all those things badly (e.g. through gobs of XML and ugly reflection based metaprogramming).
Nobody still uses them, even if they are available. In fact anything that doesn't looks enterprisey enough never goes past code reviews.
The adoption cycle for even the simplest of Syntactical features when it comes to Java enterprise is >10+ years. In some companies its never.
The saddest part isn't even that. The sad part is a whole generation of programmers have been raised, and turned to be architect astronauts without ever using something like a lambda or a closure.
The only hope for programming as a craft now is hoping Oracle kills Java(even if by mistake), and then some thing like Perl 6 comes along to replace it.
I'm not sure that will help. I see a kind of "family resemblance" between COBOL and enterprise Java. I wonder if any language that is going to play in this space is destined to become a monstrosity - destined by the nature of the problem space rather than the nature of the language.
I am aware that when I say this, I am basing my opinion on a sample size of two...
> Nonsense, the enterprise programmer of the time was programming in C or C++ and was making heavy use of macros.
C macros are entirely different in use from Lisp macros. C macros are written in a text substitution language that knows virtually nothing about C, whereas Lisp macros are written in Lisp and operate on regular Lisp data structures. Even trivial things are very difficult to get right with C macros, whereas very complicated things are often elegant and simple with Lisp macros. It's an entirely different experience. They're not at all comparable.
C programmers learn to fear macros because C macros have a lot of problems. Few of those carry over to Lisp macros.
then again enterprise software and languages are also full of sdls, plugins, frameworks, libraries and other pieces and parts that enable the same kind of power that lisp has.
People haven't really given up on the idea of building out functionality for specific portions of their codebase (and we really shouldn't) we've just seemingly exchanged one very powerful solution for a sea of different tools.
Makes me think of Cyc which is still around ( https://www.cyc.com ). Wonder if any HNers have experiences with it? I tried opencyc and was fascinated. It's a prology lisp married to a huge database of rules and knowledge. The open source version doesn't ship with the knowledge part so it's hard to evaluate if it's of any practical use. Was meaning to apply for a research license but didn't find time yet.
Lispers praise Lisp's homoiconicity but today's networked world calls for a strict separation of data and executable code as in NX bits (and yes I'm aware of the classic "Can programming be liberated from the von Neumann style?", and don't agree with it).
I don't know. I've always seen Lambda calculus as a model for computation to reason about program execution, but not as an actual implementation technique.
You are mixing up concepts. The Lisp concept of "the program is data" has little to do with modifying the program at runtime. It might be used for that in uncommon cases, but it is not recommended. A Lisp program usually is compiled into a static executable with a strict separation of data and executable code.
The mixing between program and data happens at build time in the compiler through the macro facilities. The first important thing is, that a Lisp compiler is not something which runs as an abstract process, but runs in a Lisp system and can execute arbitrary Lisp code. So Lisp macros can execute any Lisp functions - written by the user - to transform the input code, represented as data which can be easily processed, into the output code, which gets compiled.
The whole point is, that with macro expansion you are not tied to some pattern language, but can run any user-written code to process the code for output. This gives Lisp great power and extensibility. But this happens at compile-time, not at run-time.
A good example of the power of this is, that the Common Lisp Object System (CLOS) can be entirely implemented in Common Lisp. You take any Common Lisp implementation which doesn't have an object system and load the CLOS code and you have the object system available.
Well, it's not like that's a thoroughly unknown problem. eBPF (https://lwn.net/Articles/740157/), for example, which effectively relies on running user-supplied bytecode in kernelspace, attempts to solve this (and does it reasonably well) by imposing a couple of constraints on the code you supply. These constraints, in turn, allow the program to be statically-analyzed before running it.
In my experience, 90% of the cases where homoiconicity is useful are cases where the code you execute is trivial enough that you can statically analyze it, for example. E.g. you use the data to generate trivial processing code that would be tedious to write by hand.
I don't think that's the case. Template Haskell lets you generate Haskell code at compile time, including by doing arbitrary IO. This is technically unsafe but is usually okay because you're only reading local files and so forth. It's occasionally really useful for generating things like database mappings, lenses, etc.
Shrdlu is perhaps the most classic illustration of Lisp's power[1], A classic program from 1968-70 that allowed natural language communication about a micro world. It was written in a version of Lisp even more free-form than today. When I looked at the source code a while back, it's parsing of natural language involved a wide-variety of on-the-fly fix-ups and such to take into account the multiple irregularities of human language.
The thing about Lisp's power is it allows the production of complex program quickly but doesn't particularly have a standard way of gluing these programs together. The other classic Lisp work, The Bipolar Lisp Programmer[2], describes how Lisp has multiple partial implementations of important libraries and programs simply because it allows a certain type of personality to by themselves produce something that's remarkably good but doesn't encourage any particular group effort.
Lisp is certainly evidence that "language matters" but not evidence that Lisp is where one should stop.
[1] https://en.wikipedia.org/wiki/SHRDLU
[2] http://marktarver.com/bipolar.html