This is really an argument against higher-level languages in general. A complex program written in a lower-level language will certainly be more explicit, and because of that many individual lines will be easier to read. "INC EAX" is as readable as it gets, by that standard. But what you correctly call the gestalt of the system is precisely what won't be easier to see, because you have so much more code to work through. The idea that a system becomes easier to understand when its codebase is much larger is absurd. One major reason for making a system smaller is that its design becomes more accessible, and so in turn does the meaning of its parts.
In my experience, the kind of language hacking that goes on in Lisp systems is not as you describe it. Rather, it's akin to what any programmer does when extracting duplicate code into a function. A better word for this than "DSL" is "factoring"; Lisp's strength is that it offers a simple and powerful way of factoring that is harder in other languages.
I can't comment on Smalltalk, except to say that the way you've lumped it in with Lisp makes your argument weaker. Those two languages achieve their flexibility in such different ways.
This is really an argument against higher-level languages in general.
No, it's really, really not. It's an argument against deep home-grown abstractions when there are close approximations already available in the language or its community.
Hopefully you've had experience of a wide range of languages. Do you see a pattern in those languages, where, when the abstractions provided by the standard library are higher level, that libraries are generally more useful and the ecosystems are richer? When collections are in the standard library, libraries can communicate using these collections - but when they're not, every library chooses its own collections, sometimes using iterators, sometimes callback enumerations, sometimes linked lists, sometimes opaque handles, sometimes length-prefixed arrays, sometimes null-terminated arrays, etc.
This is what I mean by sociology. The more our code shares in common with other programmers, the richer our ecosystem is.
And the more our code diverges from other programmers, the more the ecosystem hurts.
Another word for "factoring" is compression. Taken to its limit, eliminating all duplication makes code indistinguishable from random noise. Human communication has redundancy and duplication for a reason.
Both Lisp and Smalltalk are able to implement 'if' as a library routine rather than a primitive. Their power to build abstractions is why I've lumped them together, not how they achieve their flexibility. How is irrelevant. Its the amount of rope they give programmers to grow their own little divergent gardens. It's how much they can encode their own idiosyncrasies - their common patterns - into their programs, and compress (aka factor) their code using those uniquely odd patterns.
I believe this rope is what prevented Lisp and Smalltalk both from becoming big successes. I suspect this rope is what might kill Ruby one day, but the amount of convergence and the extra coordination / speed of information spread from the modern web has already been enough to help it go far.
I think you've misread my statement as an argument against high-level languages, and are reacting to stuff I'm not writing.
I'm actually arguing against divergent development of home-grown abstractions when they don't have substantial wins above using what everyone else uses, even if what everyone else uses is worse - even if it's, say, 50% worse by some metric. It's also addressing the example in the original article, which is about creating your own class language in Javascript.
If Reg's class language took off, and became part of a generally accepted library - as common as jQuery or even CoffeeScript - it would be fine. But in isolation, used here and there, it's very much not worth it. And yes, I'm aware of the catch-22 here in adoption. That's the rough road every new language needs to travel, because that's basically what Reg is doing; building a new language.
Ok, if you're sure your argument doesn't apply to high-level languages in general, I'll take your word for it. In that case I'm not following it very well.
I don't think there's any "statistical significance" (using the term loosely) in the failure of Lisp, Smalltalk, or any other language to become a big success. There are too many non-technical and historical factors that can easily explain all these outcomes. Meanwhile the sample sizes are so small—consider how short the list would be of all programming languages that ever had a major turn at bat—that we have no way of drawing true conclusions about this.
For example, it's easy to make a case that Smalltalk lost to Java because (a) the Smalltalk vendors were short-sighted and (b) Java was magically "the internet language" just when the internet was everything new and important. Those are historical accidents that had nothing to do with abstractive power.
A word about the 'antisocial' argument, which I think is deeply wrong about both Lisp and Smalltalk.
It's true that Lisp and Smalltalk let you define "if". That's a measure of their power. But anyone who thinks that Lisp and Smalltalk programmers willy-nilly define "ifs" all over the place (or do anything at that level very often) does not understand the culture of these languages. There are many examples of such willy-nillyness, but that's because it's a phase most programmers seem to go through with this stuff. (I did.) Experience teaches you to be more discerning, and culture, when you're lucky enough to be exposed to it, accelerates experience.
Thus the solution to the misuse of abstractive super-power is culture and community, which work perfectly well in both Lisp's and Smalltalk's case, as their long historical records amply show. A friend of mine was part of the later wave of Smalltalkers in the pre-Java days, and he talks fondly both of how much he learned from the older Smalltalkers (over BBS mostly, since he grew up in a small town) and also how the community as a whole gradually learned to use the power of extending the language well—as well as what to avoid. So it's wrong to attribute some weird anti-social property to these languages. There's even a vibrant counterexample going on right now in Clojure.
In my experience, the kind of language hacking that goes on in Lisp systems is not as you describe it. Rather, it's akin to what any programmer does when extracting duplicate code into a function. A better word for this than "DSL" is "factoring"; Lisp's strength is that it offers a simple and powerful way of factoring that is harder in other languages.
I can't comment on Smalltalk, except to say that the way you've lumped it in with Lisp makes your argument weaker. Those two languages achieve their flexibility in such different ways.