The book How to Design Programs[1], with its "student languages", is an excellent example of how languages created with Racket can be used to achieve specific goals (or solve specific problems). In the book's case, the goal is to provide a programming environment for explaining and exploring a successively refined software design recipe.
Also, the Manifesto and Racket's documentation (and HtDP) are written in Scribble[2], another language built with Racket.
I can't praise HtDP enough (disclaimer: I've only done through chapter 4 of the first edition). Still, what I did was sufficient to fundamentally change me as a programmer.
When I first began learning how to program I started off with Python using some courses on Udacity. But somehow I found HtDP (I think via HN) and it really taught me, well, how to design programs.
It also made thinking functionally my default mode of thought. I remember hearing about FP and asking a friend experienced in FP "How do I think this way?" and he wasn't really sure how to answer. HtDP helped teach me that way and made it natural.
The only criticism I can give of the first edition is that it was at times verbose, but it is clearly written as a textbook (and is definitely suitable for self-study). The second edition appears to be more streamlined. Something that might be more of a "dive into it" kind of book is "Realm of Racket," although it doesn't appear to go as deeply into Racket's language-making capabilities and lispy features like macros.
Indeed, the first edition is draggy at the beginning. The second edition is meant precisely to dive right in and get cracking, without sacrificing any principles. I recommend people read the second edition instead of the first one. [Why should you trust me? My name's on the cover. <-;]
Also, please know that I am sincerely grateful for all of your work. Without the software and materials that you and your team labored to create -- and which you provide freely -- I might not be a professional programmer today.
I'm teaching my kids how to program using the 2nd edition because it's the introduction to programming that I always wish I had received. They love it! Thanks.
Oh my yes. I've half played with the idea of aping it for beginning R programmers, replacing some of the examples with suitably analogous data-y examples.
And on that note, be sure to take a look at Pyret, another programming language designed by this group, to address some of these student-oriented programming issues.
It is written all in JS (using Node) and can be completely run in a web browser apparently. I am not the first one here to mention it but it is very cool.
> It is written all in JS (using Node) and can be completely run in a web browser apparently.
Wow, I didn't know this. I thought it's another "compiler frontend" for Racket - essentially just a parens-free syntax for people allergic to Lisps. Instead it's a standalone language with self-hosted compiler and JS based runtime, which is even more impressive.
The features of Pyret make it the most advanced compile-to-JS language I know of. Built in unit testing and contracts, algebraic data types, pattern matching, optional typing, immutability by default - no other altjs language provides all of them at the same time (I think). It reminds me of Cobra (http://cobra-language.com/) a bit. Does it support AST-based macros? That would go way beyond awesome :)
It looks like Pyret is the result of some 20 years of work on PLT Scheme/Racket - most of the good and advanced parts of Racket but without any historic cruft. It's very impressive. I think it will need some time to mature, but I can totally see myself using it in production one day.
Pyret started its life as a #lang, which was delightful as a prototyping tool.
The demands of running with reasonable performance in a browser led us to switch to a direct-to-JS approach, and we haven't looked back since. The original Pyret-to-JS compiler has been up and running for around a year now; we're still learning how to improve and tune its performance, but it's proven quite robust.
No macros are planned, though it's not set in stone that they'll never happen. Just not a priority, and not something that Pyret's use in classrooms has caused huge demand for.
Pyret does indeed have a number of Racket's good ideas in it (especially from the teaching languages, and especially with respect to testing), along with a few new ones. However, I'd like to temper the comment "most of the good and advanced parts of Racket..." A number of Racket's advanced features, for example #lang and powerful macros, the super-expressive class/mixin/trait system, and OS-level resource management, aren't things that Pyret handles right now, and won't come right away.
Pyret's development will continue to be driven by classroom usage and curricular demands first, so it eschews, for example, the "everything should be in the language" part of Racket's manifesto. That's just a case of differing goals, which means that if you're, say, prototyping a new language from scratch, it will likely always be a safe bet to do it in Racket rather than in Pyret. (But if you're prototyping an algorithm over algebraic datatypes and want a web-based editor to try it out, just hop on over to https://code.pyret.org/editor and dive in.)
As a note on where we're maturing to, Pyret will accommodate gradual typing to mix static and dynamic checking of annotations, which is an area of active development, and something that Typed Racket is pioneering. Another thing we're thinking hard about is native JavaScript interop, to bring the lovely libraries of the Web to Pyret programmers, without forcing beginners to grok the intricacies of JS and the browser's evaluation model.
Thanks for your work in the Racket world. Thanks for Pyret too, I just do not use it ... yet. I check out your site often enough bc your work comes up a lot here.
I am curious how are you typically advising (Brown?) students to use this. I have a work computer, Windows 7, and 8GB of RAM with SSD disk and running Firefox 38.02a Developer Edition. After disabling NoScript, it took somewhere 2-3 mintues to run. This is an improvement over my last attempt, which crashed my browser a few months back, and I did not try it sense. Is this normal? Did kids have trouble loading it up in class on less powerful commodity machines, or is there something messed up with my laptop in particular?
It is super impressive you have a whole PL toolchain in browser, but does this not actually cause a barrier for your kids? I know I am a FOSS/terminal/off-the-net type and an outlier, but not having offline access would be a bummber to me.
Granted, I will evenutally get around to attempting to compiling the different compiler phases on my personal laptop, bc I do not want to do Node on Windows right now. Haha.
Still, I wish tools like this were around when I had to learn in C++ in uni, as running g++ on a Solaris box helped me along in my eventual love of Unix environments, but really distracted me with a focus on tools and less on theory and real experimentation with languages. Somewhere in between was later use of Python and REPLs. I can see why hardcore instructors like yourselves argue for Schemes, because the early focus on interactive REPLs does indulge a certain curiosity and playfulness that would have really helped me consider CS earlier in life. Again, I am not sure if inverting the order (start with theory, then focus on tools) would have made a difference in the end, but you NEU and Brown Racket guys have shown some real intensive work on CS and PL pedagogy. Thanks for your work. I browse your books often and I hope your passion continues to rub off on me. :-)
Thanks for the kind comments. Sorry to hear about the crash, etc. We haven't gotten many reports about this; if you could file a bug report the next time (https://github.com/brownplt/pyret-lang/issues/new) we'd appreciate it! [But Firefox is _really problematic_ for various reasons. That may be half the issue.]
Pyret's compiler is actually written in Pyret itself, so that it can run entirely offline. Once you have the initial page (which could be offline), you can literally turn off all networking and continue to work forever. Of course, saving files is then a problem—but I actually do this when I'm on planes (I copy my code to another file to backup).
The _first_ load is indeed a bit of a problem. We're looking into various ways to reduce that. However, you should not have had delays on any subsequent runs. We have students in other universities, and even high schools, using it, so I don't think there's anything Brown-specific at work here. So I'm afraid it may be something to do with your particular laptop )-:.
Thanks for the feedback. (EDIT: I should note that my only substantial contribution to Racket proper is bug reports and one blog post. I'm mostly just a fanboy.)
The load problem definitely doesn't jive with my experience, so I'd like to hear about it if it happens again.
I do a lot of Pyret work on a many years-old Acer Aspire Timeline X, and the slowest loads (in powersaver mode) take 25ish seconds. The lab machines my students work on take about 5-10 seconds, their laptops differ widely in (guessing a bit here) the 5-30s range, and many do work on their laptops. Students are somewhat annoyed by the load time, but this is actually not out of the ballpark for how long it takes, say, DrRacket to start (just now it took 5s on my lab machine, ~11s on my piddly laptop). So in our experience it hasn't been prohibitive, but that doesn't mean there aren't classes of machine/browser combinations that screw it up that I don't know about.
I recommend Chrom[e|ium] to my class at Swarthmore (Chromium is installed by default on department machines here), though Safari and modern IE (IE10+) have worked totally fine in practice, too. Firefox has some unfortunate interactions with the way Pyret does stack management that can make it really slow. I do have a few students who stick by Firefox and do all their work in it, despite the slowdown, so it's not unusable for them, but it is annoying.
For some assignments I do imports from Google Drive for black-box support code, and that requires connectivity, so I've gotten some minor complaints from students doing a lot of air travel (for e.g. job and grad school interviews) about that. But my students are used to having some assignments that require access to machines in the department, so it isn't a huge deal. With a Chrome App, or even just upcoming standards like ServiceWorkers, we should start to be able to cache enough/do more in the browser to make this a non-issue, even across tabs closing and browser restarts. There's more we could be doing right now with localStorage to get around this, even, but just haven't done the engineering required. So no fundamental obstacles to a better offline experience for the browser-based editor, "just" substantial engineering work.
The CLI and packaging up Pyret as an installable "binary" (really a JS blob + Node) are works in progress, but having a good CLI repl is an explicit goal.
Yeah, because those like me who mooch off of the free and open brilliance of Racket (I mean, except for paying for a copy of Realm of Racket, but that is indirect; the author has a Scheme/Lisp crush like me, but we all know that goes back to community enthusiasts and not Racket itself) get to write blog posts on the official blog while we navel gaze. You clearly contribute enough to talk semi-officially even, so thank you for your work. :-)
Jokes aside, maybe I am just conflating you academic creds with other NEU/Brown people, but maybe it is just the blog post. Your last name reminded me of Racket when I saw it; not sure why though.
Kudos to your group, or just you personally, for recommending Chromium, and NOT Chrome. I work for a university IT department, and our inventory confirms even after lecturing co-workers, I am the only one to have installed it on Windows laptops, or even Mac or Linux if you can believe the latter. Mostly everyone has no idea you can download the FLOSS fork of Chrome without the nasty proprietary bits. In the future, it would be cool to see you guys package a more focused IDE with the rage, node-webkit or nw.js or whatever is called now. You guys seem like a great example of a lang that could shine here, much like Dr. Racket is a pretty wonderful example of the robustness of Racket (although I still prefer Geiser myself).
I also see shriramkmurthi talks about university requirements about minimal install and configuration overhead. This is a great, if not just good, thing. Again, as a university IT guy thank you. However, I still have mixed feelings.
I worked for kids in a very spoiled institution with expensive new laptops, complete with full admin control for students. I get my fair share of novice programmers install a full Stata stack for the econometrics class, and along with professors, hose a 4GB or 8GB RAM laptop which "needs more drive space and RAM because my 1GB CSV dataset is so big." We tell them to write tighter code and they are incensed (seriously had students demand a more expensive version because one version of Stata had an arbitrary limit of variables in a given program, IIRC correctly, and the group of students and professors were told to us less variables, less than 60,000! I cannot imagine production distributed systems code needing that many, haha).
So, I bring up that last anecdote as a warning. I would love to help any way I can. Damned with compiled binary LLVM and Java toolchains that need admin privs and 10GB of space for library source and various dependencies, also damned with in-browser JS bootstrapped compilers that sit on an increasingly over-generalized OS in an OS.
How do you plan to handle the abstractions moving further and further away from the metal? I assume this is to teach programming basics only, right? Like Racket is, kind of? Do you worry about pushing this farther down into systems programming, or are you staying focused? I do not intend to call out or criticize one way or the other, just curious to pick your brains.
Calling out and criticism is fine! But to answer your question: for the foreseeable future, our "metal" is JavaScript, so that's what we will be finding ways to expose. We have already designed our stack mechanism to enable decent JS interop (e.g., no CPS). We want to make it possible to import JS libraries without hurting security, for instance. We've got some students right now working on integrating graphing and other functionality that we'd like to make widely usable.
Very cool, Well, I am still very much intrigued. This has been one of the most fruitful HN conversations I have had. Frankly, JS is metal is not my cup of tea, but I really like where your heads are at, so I will start reading PAPL and really playing with this.
By the way: Racket is not at all for "programming basics only". Racket, the language, is as rich a programming language as you'd like. Even the Racket team's pedagogy is wide-ranging, from middle-schools to introductory collegiate to upper-level programming languages to graduate-level programming language research.
Hmm, I should rephrase. That was, not so ironically, an over-simplification of what I mean.
When I say basics, I mean you guys get to core CS principles and notions, not basic in the simpleton sense. I see Racket as a very impressive extension of SICP wizardry, and not just the #lang/sicp. I mean building an extensible series of interoperable Lisp and non-Lisp languages for so many disciplines and domain-specific requirements is a ridiculous, daunting. PLT/Scheme Racket is something I stumbled upon when I started to research "Scheme, but modern and coherent" and "natively compiling Clojure."
I have been nothing but impressed with the return to basics means building blocks for everything, from advanced GUI toolkit and continuation based web servers. I wish more people were in the Racket community to light my way with more libraries, but that is really just my laziness.
> Wow, I didn't know this. I thought it's another "compiler frontend" for Racket - essentially just a parens-free syntax for people allergic to Lisps. Instead it's a standalone language with self-hosted compiler and JS based runtime, which is even more impressive.
You know, I was really surprised too. I had just assumed it was another language compiled out with a Racket compiler like Arc or a specialized subset learning languages built into Racket, like one of the Schemes (I mean like the lang/rsr6 or lang/plai) that are a subset for study.
Keep in mind though, after reading this article, I read the mailing list and it appears they just finishing up a functional CLI REPL. It could be fun, but they are still working on the basics of making it fun for neckbeards, who would prefer, by way of stereotype, their terminals and REPLs. At least that is what I am waiting for.
The features of Pyret make it the most advanced compile-to-JS language I know of.
ClojureScript is pretty sophisticated since it is considered production ready for commercial work and leverages much of Clojure's ongoing library development directly.
• allowing arbitrarily deep call stacks (e.g., deep recursion)
• allowing computation to be interrupted
• turning async interfaces into sync ones
This means you can't just compile altjs function calls to JavaScript function calls. That's the work that @jpolitz is referring to. The last we checked, ClojureScript did not address these issues as well (though it does other awesome things!).
I think the PLT group does amazing work in general and in education in particular. The well designed teaching languages hide complexity while providing power for expressing important concepts. And nobody makes installing a full blown IDE and versatile tool chain easier for beginners.
My emphasis was on an a Blub->JS alternative for production. I am not sure Pyret is a priority alternative for a green field project due to the language's intended purpose and the directions in which that purpose is likely to drive and not drive its development and tooling. That is to say that features are likely to flow Pyret->Clojure faster than Clojure->Pyret.
Absolutely. To be honest, we don't want Pyret to stay in the "education cellar" forever. But, we need a set of coherent design constraints, and education is a good one because it prevents premature cruft.
The current stack treatment is pretty elaborate for pedagogic reasons. There's absolutely no reason we can't just turn it off for programmers who are willing to write short-running computations, for instance (as production developers are willing to do)—that would be a "#lang"-style configuration. The tooling would, of course, be the bigger issue.
Overall, I'm happy for all good altjs projects. The core Pyret team burnt their fingers on a lot of aspects of JavaScript and see clearly its difficulties for education and even for development.
By the way, this is one of the reasons we're putting a lot of effort into JS interop. This isn't really important for education but it is for development—and in particular, development for education.
I think tooling is only theoretically surmountable under the current incarnation of PLT and its culture [and I am not saying that this is a bad thing because that culture is extremely valuable].
By which I mean that PLT culture does not spin off the right tooling in the right direction relative to typical production operations. Racket works well with DrRacket and sorta with Emacs: compare Geiser or Hendershott's M-x racket-mode to Cider. Good luck finding out of the box integration of Racket with Eclipse, Visual Studio etc.
The reasons include, I think, the fact that consultants on mutli-million dollar contracts are not going to start with Racket. This means that 1000 hours building a tool for Eclipse that saves 10% on a $4,000,000 contract never happens. What happens instead, and it is a good thing, is that people tend write tools that solve the problems of academia, including implementation of research topics at one end and producing well formatted text on the other, and of course pedagogy on the third end [maths are not my strength, apparently].
As an example, listening to the Flatt interview, the sub-module system makes sense. The extra level of indentation for namespace construction though feels kludgey from a typing code into a box perspective because top-level concepts from the perspective of the enclosing namespace are indented - I look and ask myself, did I miss a right-banana?: The sub-module syntax is at odds with the core principles of pretty-printing as a tool. That sort of design engineering isn't a priority for PLT [and I am not saying it should be, only that it makes moving out of education less likely].
On the subject of tools: I'd like to clarify that Scribble was not designed to solve academic text-formatting problems; it was designed for Racket documentation. It's fair to say that we subsequently improved Scribble in academic-friendly paper- and book-writing directions, but like so much of what the current Racket developers do, it was motivated by programmer experience rather than academic tasks or credit. The current developers can only work so fast, but I hope Racket continues to attract more contributors who build more tools.
On the topic of parentheses and submodules: I can understand why some consider parentheses to be an obstacle, but I'm puzzled that you've singled out their use for submodules. Syntactic forms in `#lang racket` use parentheses, and I think they've worked well in practice for submodules. That said, submodules are most powerful when you don't use them directly and instead use (or create) constructs that are implemented as submodules. For the example you have in mind, a new syntactic form may well fit better --- and maybe should be one of the many constructs that are available out of the box.
What was the motivation to start from scratch rather than compile at least a subset of Racket to JS? I imagine Pyret was a massive amount of work. Same effort could've brought Racket to the browser.
---
Big thank you to the PLT group for HTdP that introduced me to programming and PLAI class that taught me PL hacking and ultimately landed me a job writing compilers
We've already compiled a very good chunk of Racket to JS. It's called Whalesong (http://cs.brown.edu/~sk/Publications/Papers/Published/yk-wha...). The initial version of Pyret was in fact built as a #lang in Racket, and used Whalesong to obtain an in-browser implementation.
We ran into two major problems with this.
1. It was really slow. Whalesong faithfully implements Racket, including delimited continuations and all sorts of other fun stuff. But it suffers in performance. See the paper—we went through three different implementation strategies. But for all of them, Pyret proved to be too slow.
2. We wanted an entirely in-browser experience. The earlier Pyret implementation used Racket to do initial compilation, and Racket itself is not designed to run in the browser (it depends on its own virtual machine). Though Whalesong went a long way, it could not go all the way without substantially reimplementing Racket, which is an enormous enterprise. Absent all that, we would always have to rely on a "compile server". This was an obstacle for us in a previous system (http://www.wescheme.org/) so we didn't want to repeat that experience.
Ultimately, however, bringing Racket to the browser was not what we were setting out to achieve in Pyret. Rather, we were out to build a new language based on what we've learned from (a) teaching, (b) building systems, and (c) doing various projects on scripting languages. Pyret is our attempt to condense all that experience into a language that represents our needs well (it's still a work in progress). It just so happens that one of our (externally-imposed) constraints is to run in the browser with minimal server support.
In terms of effort, Whalesong was about four years of work by primarily one person, and it's still nowhere near done (because of how big Racket is). The initial Pyret was about 1.5 person years (though some of that was also spent building a peer-review system: http://cs.brown.edu/~sk/Publications/Papers/Published/ppkf-c...). The new Pyret is about two person years, and much farther along than the original #lang-based Pyret was. Indeed, the new Pyret is solid enough to be used in several classes without noticeable problems. So no, I would not agree with your assertion that the "same effort" would have brought Pyret to the browser.
Effort stats for both Whalesong and Pyret are impressive and humbling. Manifesto actually touches the importance of tackling "assemblers" including JavaScript, so I hope someone's actively researching the problem: performant Whalesong perhaps, or an entirely different approach.
Whalesong is moribund for now. The Pyret intermediate language is actually a really good compilation target for functional languages. The problem is that Racket has a lot more stuff than Pyret (some of which are discussed in the paper, others are things like delimited continuations). There's certainly a Racket-lite that drops these features, is still a very full language, and would compile very nicely to Pyret. Anyone want to help us build that? (-:
That was my suspicion as well: there could be a very usable and rich subset of Racket that could target JS without significant performance degradation. Sidestepping the deeper darker corners like I think ClojureScript does with Clojure? Herculean effort already it becomes even more daunting when you start picking up features to drop only to discover just how much of Racket depends on say delimeted control machinery. After all Racketeers have been eating their dog food so most if not all their PL research found its way into the design. Bitten by their own success :) The manifesto takes admirably healthy stance here: problems offer opportunity for research. Alas, when u consider the time it takes to build smth like Pyret or Whalesong your pool of researchers and implementers narrows pretty much to academia. Unless of course there's some benefactor who's willing to assume the expense and allow the time
We have long-term plans for integrating some kind of rewriting system. Justin Pombrio is doing really interesting work in this direction (e.g., http://cs.brown.edu/~sk/Publications/Papers/Published/pk-res...). We are rethinking what it means to be “hygienic”, so it'll be a while, probably. But it's a thing we're taking very seriously.
First time I'm seeing Cobra. Of course, there's a 100 altjs languages out there. It's certainly cute. But every Cobra file seems to begin with "class". I have no idea why I'd want to subject a poor child to that nonsense.
Just to be clear, the flip side of Racket's 20 years of work is it is _very sophisticated_. Pyret has nowhere near the same sophistication.
Cobra compiles to CIL and runs on Mono/.Net runtimes. I think its philosophy is close to Pyret: they both seem to be taking the best ideas from different languages and trying to integrate them into one nice package. There are some similarities in features chosen, but I'm not working with either, so I can't do a deeper comparison.
The platform is a pretty big difference. One of the major problems we run into in the education space is that many schools _cannot_ install any new software. Running on the browser is pretty much the only thing they can do. We created WeScheme (http://www.wescheme.org/, http://cs.brown.edu/~sk/Publications/Papers/Published/yk-wha...) precisely for this reason, and Pyret is also engineered around this need. If you require a compiler and/or runtime download/install, you are excluding yourself from numerous schools, _especially_ poorer ones. We're really concerned about not amplifying these problems!
I read about that a couple years ago when I saw it on the front page. It's a really beautiful language, and a lot of work seems to have gone into forgetting crufty patterns that don't serve newcomers to programming. It's a real breath of fresh air!
I sense the whole area of domain-specific languages is due for a belated renaissance.
Everybody shirks away from it, but most languages are not very good for solving most problems and the standard libraries bloat out as more and more gets crammed in.
I don't think in a commercial context we can all step back and say "to solve this problem, let's first design an appropriate language" but as a means to define the problem better, or to encapsulate learning we've made in building a prototype in a general language, it can be very powerful.
If you aren't able to best describe the problem space, how do you intend to solve a problem within it?
The problem with Lisp-like DSL construction, is that the interface doesn't play well with many populations of domain experts. Even Smalltalk, with its syntax designed to be usable by children can fulfill the technical side of constructing DSLs easily, but still leaves something to be desired when it comes to domain expert use. (Various Smalltalks had their rough equivalent to #lang.)
(I know there is at least one Ruby based DSL which made some inroads to mobile device software testing.)
Basically, DSLs tend to only get used by experts who are already programmers or otherwise technically disposed. Maybe we're getting to the point where rudimentary coding is just going to be a new kind of literacy?
This is a criticism of Lisp-based DSLs but not of Racket-based DSLs. Racket actively enables you to impose different, non-parenthetical surface syntaxes on the DSL. It is not a "this sort of works if you arrange it carefully _and nobody screws anything up_" thing like in Ruby, it really is an explicit language definition.
Scribble is another great example. It's amazing in its own right, and by being embedded into Racket, gets all the Racket benefits: for instance, separate compilation! (Thirty years and waiting, LaTeX.)
I agree with Page 1 alot. You can teach a kid to count in 1s and 0s, but that doesn't necessarily follow that you can teach him to code in machine language.
Even the pros' have to use tools like hex representation to make machine language programming easier.
For some time, I've daydreamed about implementing R in Racket ("Arket"?) as a means of shining a flashlight in all the dark corners of my R knowledge. Then I realize this would take me a heap of time and it goes back to being a daydream.
Also, the Manifesto and Racket's documentation (and HtDP) are written in Scribble[2], another language built with Racket.
[1] http://www.ccs.neu.edu/home/matthias/HtDP2e/
[2] http://docs.racket-lang.org/scribble/