One thing is worth mentioning here: Parenscript is not a CL to JS compiler, but rather a way of embedding JS inside Lisp. You're not writing your PS code in Lisp and you don't have the full power of CL. Over time, people have added more Lisp-like features (for example, keyword parameters), but it's still very much a sexp dialect of JS.
On the surface this seems less cool, but it has advantages that would be hard to achieve with a full Lisp->JS compiler. The generated JS is very efficient. Plus, it's very readable, so you can use all the standard tools for debugging etc. These things are critical for application development, which is why you see full compilers pop up more in the research world and not so much in business.
Neat, but does this really actually improve web application development (the client side of things)? You still have to deal with HTML and CSS and the DOM.
Yes. Not Parenscript as such, but the design style that it fits into can hugely improve web app development. Lisp being Lisp, you can build up natural constructs on the server side for the app you want, then generate your HTML, CSS, and JS from these. Since HTML, CSS, and JS are represented as Lisp expressions, you have the full power of Lisp macros to raise the level of abstraction of your code. Essentially, the browser becomes the target platform of your own compiler and you write your client-side code in a high-level language that is suitable for what you are trying to do.
What are the advantages? Your code is higher-level, so it's clearer and shorter. You get to do most of your work (client- and server-side) in a single powerful environment. JS has become your assembly language, so you can make it low-level and efficient. This is the way out of the tradeoff between abstraction and performance in the browser. (We found that the higher-level JS libraries and frameworks are too slow because they make you pay for all that abstraction at runtime in the browser.) Oh, and it's easy to do code transformations: we wrote a profiler in about 50 lines of Lisp that works much like Firebug's but runs in all browsers.
Another advantage is that you can compile a different version for each browser that you support (no more client-side browser detection). It turns out, though, that browser incompatibilities go way down once you eschew complexity in the browser. Low-level and simple things are much more likely to work the same in different browsers than fancy stuff with CSS or tables.
There are at least three fallacies here. First, appeal to authority. Second, straw man: nowhere did I claim "to abstract away all the nastiness of dealing with browsers". (I won't repeat what I did claim, since it's just a few lines away.) Third, argument by selective example. What you call "this" isn't representative of what I'm talking about, although it does show what html and css sexps look like.
Still, let's not get into an argument, I'd rather trade thoughts. The idea of abstracting away from the browser entirely seems attractive, but turns out to be impractical once you get to know the problem better. It's like abstracting away the network. So for me it's become clear that there's no way to get away from the browser altogether (except by not building HTML-based apps in the first place). That doesn't mean we can't significantly improve matters by building appropriate abstractions. As I said, we can reduce duplication, increase clarity and efficiency, and so on. The environment I'm describing has provided more of these benefits than anything else I've seen so far.
Re Joel Spolsky, I find most of his stuff tiresome and overblown, but he occasionally comes out with a great idea. What specifically did he say that you find relevant?
I was asking about Joel's article because you used the phrase "JS has become your assembly language", which he also mentions. My comment regarding abstracting away browser nastiness was in response to my original comment, "You still have to deal with HTML and CSS and the DOM", which you didn't address.
"The idea of abstracting away from the browser entirely seems attractive, but turns out to be impractical once you get to know the problem better."
In fact, I know the problem very well, and can tell you that this is simply not true. Unfortunately I can't give you any more details right now, so I'm sure you'll think I'm full of shit.
I'll admit that a) I'm biased, and b) I haven't looked at Parenscript in depth, but from what I've seen I don't see it solving any problem well enough to get non-Lisp geeks to use it.
I was asking about Joel's article because you used the phrase "JS has become your assembly language", which he also mentions.
Ah, no, I didn't get that from Joel, it's just the obvious analogy if you're generating low-level JS.
my original comment, "You still have to deal with HTML and CSS and the DOM", which you didn't address
I thought I did address it. You can generate these things from something higher-level which is much easier to work with (but not so abstract that you don't still have to understand what you're generating). I don't claim that this is a unique approach but it is certainly an area of strength for Lisp, and Parenscript deserves to be better known.
Incidentally, in the app I work on we don't deal much with HTML or CSS. We keep those to a minimum and do almost everything in Javascript. We even try to avoid the DOM when possible and do as much as we can in pure JS (for example, using arrays rather than childNodes). This is the only way we've been able to get acceptable (or nearly acceptable) performance in the browser.
In fact, I know the problem very well, and can tell you that this is simply not true. Unfortunately I can't give you any more details right now, so I'm sure you'll think I'm full of shit.
I don't think you're full of shit. I'm describing the best solution of the ones I've seen so far. If you can do better, I look forward to hearing about it.
I remain skeptical, though, that HTML/JS can be fully abstracted away in a way that doesn't leak (now there's a term I did get from Spolsky) and still yields usable, performant web apps. There's too much going on there that's too essential to what a web app is. Take for example that html sexp you quoted. There's a lot of information in it. However you represent it, you still need a way to say that you want a 1px border, a certain font, links that get bigger when you mouse over them and so on. There's a whole model there which can certainly be abstracted into something else, but that something else won't necessarily be simpler (it will, however, be non-standard). My feeling is that there isn't much "compression" to be had in the general case. But I'm willing to be proven wrong.
" However you represent it, you still need a way to say that you want a 1px border, a certain font, links that get bigger when you mouse over them and so on. There's a whole model there which can certainly be abstracted into something else, but that something else won't necessarily be simpler (it will, however, be non-standard)."
The thing is, is that really what people who are building applications need to figure out? Yes, they probably need access to changing these properties, but the vast majority of what you are doing when you are writing an actual application on the web is far more complicated then setting a few class properties on some divs.
It isn't so much about compressing this, as it is about throwing out this broken architecture. HTML/CSS are great technologies for page layout. In fact, I have a perverse love for them which I am unable to explain, let alone understand. But they suck when it comes to dynamic, application like behaviors. We need a new technology. And yes, while it won't be a w3c or whatwg standard, if its open source, who cares? I, for one, am tired of relying on browser manufacturers to get things done.
Hmm, we're talking in the same general space but miscommunicating on details, so this discussion may need to wait until you guys can provide specifics of what you're working on. But let me try to clear up a couple points.
The thing is, is that really what people who are building applications need to figure out? Yes, they probably need access to changing these properties
You answered your own question. Yes, they need this, along with a lot of other things. That complexity is intrinsic to making a web app and therefore is hard to completely abstract away.
Of course I do not exaggerate the importance of div borders. My point was that even the sexp your colleague cited as a counterexample contains a lot of information - not just browser nastiness or crud, but useful information, which any model you try to replace HTML/CSS/JS with will still need to represent.
It isn't so much about compressing this
When I said "compression" I didn't mean it literally, as in gzip, but metaphorically. A good abstraction is a kind of logical compression inasmuch as it replaces a complex, detailed model with a simpler one that contains the same information. It does this by throwing out non-essential details (like duplication). Now, if your goal is to build a general abstraction ABC on top of HTML/CSS/JS, ABC must be much simpler in order to justify its learning curve and the fact that it's nonstandard. (By "nonstandard" I simply mean unfamiliar.) That means you're going to have to find a lot of "noise" to throw out. My skepticism is based on the fact that I don't see a lot of such noise in the general case. A thing can be messy and yet still difficult to abstract over. But I'm not saying it can't be done, and good for you if you can do it.
The approach we take with Parenscript is more modest, I think. We're not building universal abstractions for web apps, just for our own. That means we can abstract away from anything we don't need, whether or not it's needed by others. That and the fact that Lisp gives us a homogeneous way of representing all the HTML/CSS/JS pieces are two very big wins. Is it enough to get non-Lisp-geeks to use it? Probably not. But I don't think that's why most hackers take up something like Lisp in the first place.
I think the problem is we have two very different solutions to two different problems. Parenscript seems great for the things you're describing, but those are the things I happen to not be interested in.
I'll be sure to let you know when we have something more to show, I'd be interested to hear your opinions of it.
you're missing the point. This is not for non lisp users, it's a way to make javascript and client side development more bearable for lisp users.
As far as the abstractions, you will need to write some ugly DOM code at first, but then you hide away the ugly low level stuff inside of a package, write a couple of macros and you move on to writing the meat of the application at a higher level of abstraction.
To make an analogy, think of starting with C, implementing Python in it and then writing your application in it.
It's clearly not a silver bullet. Basically it looks to me like the main advantage is that you can write client-side code in Javascript using s-expression trees instead of C-like syntax(statements and binary operator expressions).
On the surface this seems less cool, but it has advantages that would be hard to achieve with a full Lisp->JS compiler. The generated JS is very efficient. Plus, it's very readable, so you can use all the standard tools for debugging etc. These things are critical for application development, which is why you see full compilers pop up more in the research world and not so much in business.