I don't get it why Clojure and other Lisp tutorials throw things like this in face of beginners:
(reduce + (map (comp inc inc) (range 10)))
instead of just using things like the "->" macro to make the damn thing just as in any other language:
(-> (range 10)
(map #(+ % 2))
(sum))
...it's as if they purposely try to scare people away from Lisp! Some kind of smug superiority makes them wanna make sure no "lesser mortals" want to touch their beloved language. (The added benefit would be that later you can "blow their minds" but showing them how to write "->" themselves and instantly introduce them to the power of macros!)
This is almost close to showing a code snippet like this to someone trying to learn Haskell:
Because -> is not a typical functional idiom, it's a macro specific to Clojure, and anything you can do with -> you can also do in the standard functional way, and beginners are better served by understanding the way a lisp works rather than only get taught syntactic sugars. Keep in mind that -> as a macro does not evaluate its arguments the way a function does, which complicates the understanding for a beginner tremendously, since Clojure is actually an eagerly-evaluated language.
Ha, yeah, I get where you're coming from. And I toyed with putting a threading macro example inline, and in the end put a footnote in instead.
My reasoning was that basically, regardless of the threading macros, you are going to have to learn to read/write s-expressions as-is for a couple of reasons:
1. The threading macros won't be the best way forward in all cases.
2. You're going to need to read other people's code, which wasn't using the macros.
Overall, I was trying to make the point that it is a real hurdle (at least that's how it felt for me), but worth getting over. As a week or so later, you can read s-expressions with relative ease. That it's worth making the effort, because they are pretty awesome. Maybe I just didn't communicate that very well.
Good point. My code is littered with threading macros all over the place, and it can really reduce ))))))))))) to ). But I wouldn't have been able to use it effectively if I didn't know how s-expressions work.
This is subjective and dependent on the background. I, as a beginner, found the original expression easier to understand, whereas, the "->" macro rewrite seems puzzling. Specially (map #(+ % 2)) looks as if the values are summed and then % 2 is applies to it.
As a Clojure/Lisp beginner, I can't say I agree. I don't know any language that works like your example, except maybe shell script with its pipes. Without variables, that expression just seems too "magical". Nested function calls, on the other hand, are common everywhere.
While they might make the flow clearer, I don't think the swiss arrows fix what I perceive as more confusing for a newcomer.
The problem is that -<> looks like a regular function call, but does things that a function call can't do. In a language without macros, which are most of the mainstream ones, <> would have to be a language feature, and then it's not clear why you'd need to use -<> to 'activate' it.
Now, I've read enough about Lisp and AST and such to understand what a macro is, but the concept doesn't map easily to mainstream languages.
...that's how chaining method calls end up working in most OO languages (take the Scala example in OP's article). And also in Javascript (I can't remember if there is something like `range` in js right now...):
[1,2,3,4,5,6,7,8,9,10]
.map(function (x) { return x + 2 })
.reduce(function (a, b) { return a + b })
so the "." in OOP ends up working like "->" or "->>". Heck, you can make the dot in OOP code behave like anything, even like monadic bind (`>>=` or `do` notation in Haskell), if you do OOP-style-monads in JS: https://www.youtube.com/watch?v=b0EF0VTs9Dc
(off-topic: this is also my main problem with OOP... the "." can actually end up meaning anything practically, as program logic is concerned, it's like some kind of "infinite operator overloading" :) )
But (conceptually) the "." in most OO languages is something that is part of the object it's being applied to, while the -> is something external that one applies to the subsequent expressions. At least, this is what I perceive by looking at the code, and in my opinion, the two concepts don't map intuitively.
Why other Lisp tutorials don't teach -> is, firstly, that other Lisp's don't have that, except as some add-on library feature. Secondly, it's a syntactic sugar that has quite a bit going on under the hood: implicit parameters, a lambda function being expressed as #(+ % 2) with this % thingy, and return values of forms being passed as arguments to other forms that are not syntactically enclosed. You have to understand that (range 10) is just called normally, but then its return value becomes an extra invisible argument to the (map ...) form.
Newcomers should learn the underlying ordinary evaluation first.
Totally agree with your point, but just a quick note: using the thread-first -> macro won't work in this case, instead you'd use the thread-last macro ->>. Also Clojure doesn't have a sum function, so you'd actually use (reduce +).
only in this case, it should be ->>, because -> inserts argument into first position, but you need it in the last... And you can omit parentheses around functions without additional arguments (sum in this case)
EDIT: as observed by others, `->` should actually be `->>`, sorry for the typo, but unfortunately I can't edit the original comment anymore. Also forgot there's no "sum" already defined in Clojure, so replace that with `(reduce +)`
This is very similar to my experience. There's definitely some friction getting started; the odd using/require macros for importing (which switched sometime in the last 2 years, which means a lot of out-of-date SO answers); the fact that most docs recommend using emacs which, if you aren't already a user, is not going to be easy to deal with; the parens.
But it's amazing. I do really miss types, but found that unit tests + Schema[0] serves my needs; they provide some type-ish enforcement, and do a terrific job documenting my code so that when I return to it months later, I have some idea of what's in all these map structures.
Another unsung feature is that emacs + paredit + s-exprs + repl means the actual mechanics of writing Clojure is wildly better than any other dev environment. The ability to operate confidently and consistently on semantic units of code vs. text is really amazing and I immediately miss it when I switch back to JS or Python. I'm sure other Clojurists felt they had already experienced something like FB's Prune when it was discussed here a few days ago.
Not to mention learning a Lisp really does increase your understanding of what's possible. I've seen others recommend holding off on playing with macros until you've become quite familiar with the language, but I couldn't disagree more. Play with them right away! I have perhaps ~200 hours of Clojure under my belt -- barely anything -- but I've hacked together a few useful macros. Though I barely understand how they work when revisiting them, it definitely increased my understanding of what's possible, and what's missing, in other languages.
I spend most of my time in emacs doing elisp and python, where everything is smooth, small and stupid. I just got back into Eclipse/Java. And the culture shock is so strong I had anger attacks for two days.
The Enterprise world is quite weird, they don't really care about the code, it's just an artefact, the IDEs reflect that, the 'UX' too. They don't really want you or understand how to manipulate code fast. It's loads of declarations that gets in the way of thinking, all inside ergonomically poor tools that are supposedly necessary to survive (Types are nice, but Generic OOP still feels like a burden compared to functions + HM). They want documentation, they want an asset, that's how they 'manage' complexity in the long run. All of this so you can mutate tiny bits of arrays... Crazy.
It also shows how language and standard library design matters. Python APIs are often tiny, everything is in this language, you rarely have to fight to remember, it falls down pretty nicely, and in case you have doubts you're one repl/dir/help away from finding out. Different culture.
Of course, in other paradigms, what is taking so much energy would be done in a 3rd or maybe less, so you don't need all the ceremony...
Guess what, reality, I have to find a job, IT consultancy hires as always, JEE is still the main thing in enterprises. If this fails or maybe later, I'll try to switch towards Scala, Clojure, F#.
What's your line of work ? how many other languages do you know ?
I use clojure for 4 years now and I think I had to write macros only twice. In general, I'd advise against them and be wary of libraries that try to solve a lot of things with macros, the main reason being they are not easily composable and make things implicit that should be explicit.
The parens become less of a problem the more you know about clojure. You will find ways to traverse a map with very few parentheses that will make you grasp the code very efficiently. To this end I would recommend anyone starting clojure to do a few clojure koans each day and check their solution against the best solutions. www.4clojure.com
Finally, the tests (assertions via assert, :pre and :post) are indeed THE thing to use for both documentation and type checking and it allows to have kind of static typing where you want it. Of course, exceptions will only occur during run-time but your tests check for errors during run-time anyway, so why force someone to do type checking at compile time?
Why? Macros are the single most important productivity feature in Lisps.
Macros allow to implement eDSLs trivially, in the most robust way possible. Yes, you need to follow a certain discipline, but once you get it, you won't ever want to touch a language without macros.
Most of the macros in libraries were saving a pair of parentheses at the cost of making code more obscure and, well, less explicit. A few co-workers also liked macros alot so they did stuff that could have been done in functions.
I suppose my experience is that clojure's core macros are sufficient and/or I never saw a good eDSL that would have been worse without macros. (Input is welcome)
Also, I don't feel I missed out on something because the biggest productivity features for me are being functional, maps-as-objects + dynamic typing, destructuring, emacs (yeah I know), immutability, threaded execution via pmap, Java libs.
However, I suppose these productivity features depend strongly on where you come from (for me: Java)
> Most of the macros in libraries were saving a pair of parentheses
This is an epic misuse of macros.
> and/or I never saw a good eDSL that would have been worse without macros
eDSLs should almost never be implemented without macros (unless they're message DSLs).
Macro, essentially, is a compiler. The only alternative is to implement an ad hoc interpreter. And compilers are much, much easier to write, debug and maintain than interpreters - for some very trivial reasons. Compilers can be split into sequences of very simple stages, each doing something simple and comprehensible. It's easy to debug, easy to insert additional steps in between, easy to cut off at any point and replace the backend with something totally different. Interpreters are deeply tangled, naturally, there is no way to modularise them.
> the biggest productivity features for me are being functional
Using macros and eDSLs is another huge step forward in productivity that will dwarf everything you liked so far in FP.
I've been very successfully applying the data > functions > macros guideline. Yes, this makes eDSL's look very clojurey, but in practice I've not seen this to be a problem.
That is: prefer a data-structure based DSL (eg Datomic queries) if possible. If not, prefer function composition. Failing that, macros to the rescue.
This is not any different from a macro. You still translate your DSL into your host language AST, with the only final step different - runtime eval instead of a compile-time macro.
All my macros look like (defmacro something (dsl) (compile-dsl-whatever dsl)). You can always pass the same (compile-dsl-whatever dsl) to (eval ...).
My understanding is that the Clojure community discourages the use of macro. The preference is to use data over functions, and functions over macro. The reason is that macros do not compose well.
Yes, macros are good for developing DSLs, so they are mostly used in developing libraries, but most application developers do not need to use macro. The wide spread macro usage probably contributed to the fragmentation of CL community. Taking such lessons to heart, Clojure library authors tend to reuse commonly accepted DSL forms, e.g. hiccup is the DSL for writing HTML, and most authors have adopted it when developing their own libraries. I think this is a good thing.
Then the entire community got it all wrong. DSLs are much, much more composable than functions. You can easily compose just parts of DSLs, building new DSLs on top by combining semantic properties from the existing DSLs.
It's just a very different way of thinking about programming in general, and many people just don't get it.
Macros look like the single most important productivity feature in Lisps only when we are comparing to languages that have borrowed almost everything else, so that we are discounting those other things, consciously or not.
I'd trade any language without macros to the same language, but with the proper macros added. And then I'll get all the features I want - lambdas, garbage collection, numeric tower, lazy evaluation, any imaginable type system, whatever else (of course, there are some limits, I would not be able to get an image-based system and a REPL if the host language runtime does not allow it).
It's a nearly infinite flexibility, no other single feature or a combination of language features can do this.
Note: you don't need to commit to Emacs to get the productivity benefits of Clojure.
Eclipse with the Counterclockwise plugin (and I believe IntelliJ with Cursive too) also give you comprehensive REPL, s-expr manipulation and paredit features.
The benefit of CIDER (the Clojure Emacs development environment) is that it now has a s-expr based debugger, whereas Cursive's debugger is still call stack based. So you can traverse the AST like in any other Lisp debugger and evaluate and manipulate s-exprs as you go.
> Another unsung feature is that emacs + paredit + s-exprs + repl means the actual mechanics of writing Clojure is wildly better than any other dev environment.
Not if you take commercial Common Lisps devenvs into account.
And I dare to say that even Macintosh Common Lisp had a better developer experience than current Emacs based tooling, but lispm can correct me on that.
I've asked on StackOverflow, got shut down violenty, so I try to ask here: how do developers in large(-ish) projects deal with the lack of tool assisted refactoring? (Renaming a field in Java, I am 100% sure that every call site is found, changed or otherwise the codebase does not compile. The catch-all clause of dynamically typed language users seems to be that the test suite should catch everything, but looking at most projects, it's outstanding to even see 40% test coverage.)
I've never really needed it in Clojure. The code is that much more expressive that overall LoC is dramatically reduced (thus more easily viewed "in the whole"), and following a proper functional approach means good functions should be largely interchangeable and not dependent on any global state. So on the rare occasion I need to move something around, it's just meant a quick copy and paste and tweaking a few references.
Yes, Clojure is dynamically typed, but I think you're missing the gains you get from FP and expressiveness in terms of reducing labor.
I was trying to find time for Clojure, but I had always had doubts - exactly as the grandparent commenter...
He touched upon what I think is essential. The biggest things to lose by giving up Java is the tooling. Using the IDE I have many possibilities to learn about code, without ever needing to run it. For example by using find usages/implementations. The types on methods often give out hints about the big picture. Like, if a method passes a JAXB context, than I know I'm in the infrastructure layer. If I'm looking for a business rule "how is the price calculated", I can very quickly cut off a large portion of code which does not do it. After few iterations, it's almost certain I will get to the relevant code part.
Then I typically launch an application and put a breakpoint, to test my assumptions. After that, I have a lot of confidence about how the application behaves in that spot, even If I saw it for the first time.
If anybody asks me "Tell me if we can extend the functionality of calculating user score to include expert rating", I can provide a good effort approximation, even in a never seen before application. I just know, that given the tooling, I can filter out unrelated code parts and concentrate only on the relevant one.
The above concerns only code comprehension of a large project. The tooling does give you also power to do macro level restructuring, like breaking a large monolith into smaller modules. The compile time errors are like a high accuracy automatically generated todo-list.
Since it takes so little effort to make large changes, you gain the ability to do it multiple times and refine. If I broke something into two modules and a dependency cycle came up, I see it right away and can do everything again, a little bit better with the extra knowledge I got.
I have never seen anybody doing the same in Ruby, maybe because I've been working only about two years commercially in it, but a lot of times I've seen something quite opposite: introducing more hacks decreasing code quality, just to work around some conceptual error. While working in Java, I have seen a lot of situations like "we'll doing a hack here and the real solution will come with the next big release" and really the solution does come.
High level changes typically break a lot. With tooling, you gain confidence you can fix them, without introducing a lot of regression bugs. In Ruby, I saw the people just give up and leave monkey patches around. Then they just leave the project for the next unfortunate programmer.
So, back to the topic. You say that the change name refactoring is very simple in Clojure, because the impact of change is local to the method. My question is: How does Clojure tooling aid You in changing a place with a global impact? Like, changing the argument list of a very often called method (e.g. the method to serialize a structure to string to require the caller to provide an explicit encoding)? Can You, right after doing the change, quickly estimate how much work is there to fix all affected places? Do You have confidence that the list you provide is complete and doesn't miss anything critical? How certain can you be without running the code, only analyzing?
1) That's up to the tests, just as in any language.
2) Some of the changes you describe are no less trivial to experiment with in an environment with a REPL. I routinely trial changes to small functions simply by running them in the REPL as discrete units before attempting to re-start an entire server project.
3) Good FP strategy rarely means dealing with anything with a global impact. About the only exception is client-side ClojureScript, where keeping a single "state" atom is largely idiomatic, and even then, actually using it is a last resort action reserved for things like routing and user session, preferring instead local state unique only to a single file for anything else that requires it.
I think this is a case where you need to experience just how big the difference is between an expressive FP approach and something like Java is to really understand why it's not a problem, and indeed, amounts to something like a category error.
When a Java programmer asks me about refactoring Clojure features, I can't help but think about stuff like how the entire messaging system for the app I worked on over the summer fits on maybe 3 or 4 printed pages of code and only took a week and a half to complete, compared to say, EnterpriseGradeFizzBuzz ... The tools you describe sometimes don't exist, it's true, but that's because no one needs them. They exist to automate tedium and complexity that is largely idiomatic to Java and other similar OOP languages, but not expected, desired, or necessary in other paradigms of programming.
I know it's hard to believe, but there are other ways of doing things. We do alright, I promise.
After I reread my own comment I realized it sounded much more harsh than I wanted. Good you were not put off by the tone. Sorry for that.
I have to admit that my biggest pet peeve with working in Java projects are those high impact changes. I try as much as I can to propose "extending" changes, but working in a team means someone certainly at some point will propose "changing" how something works, without ever realizing all the consequences.
If by using Clojure you get the same benefits of what the best OO practices can give, than I'm willing to give it a try.
I do remember a presentation about working on immutable snapshots of reality, it was something along the lines "How do you normally check if a runner had both feet above the ground?". The example there was, that a typical imperative programmer just checks the first foot and then the other. Naturally, the second check happens some time after the first one. The selling point was that, in Clojure, that you operate on a immutable snapshot of a runner in time, so the feet are by default in the same point in time.
So, I'm liberated from thinking about a whole category of potential errors. I liked it a lot.
There are some other categories of potential errors as well. I mentioned them in my comment and you cleared that up for me.
Still I have another doubt, regarding a typical code reuse scenario. A simple example follows.
Let's say a programmer needs to ask the user for input. He writes a code, which pops up a user interface dialog and wires up an action to the buttons callback. All is fine. As the time passes, there is a need to provide more questions to the user. The dialog code grows. Someone files a bug, that, in a multi-monitor setup, the dialog shows up always on the wrong screen. The bugs get fixed. More dialogs are introduced. Another project starts up and wants to use the nice UI code. We designate it for a separate library. The first project must isolate and make everything generic. The requirement is that both projects must work on top of that library and the first one cannot loose any functionality.
This is where the tooling excels most. You just make the duplicated UI code look the same and use the "extract method" refactoring. The IDE tells you "I just found 5 duplicates, should I replace them all?". You answer yes. The IDE even tells you if there are any potential side effects. If there are none, you proceed. Then you use "use interface instead of method call" refactoring. In that way you can nicely extract contract between the client code from the library code. You move the stuff around, so that the library code lands in other project.
Then, You can just wire the library as a dependency in the Project B. It happens, that some needed functionality is not exposed. So you change the library and immediately see the impact on the A Project.
The case here is: you do not have to understand the A Project as a whole, since the tooling takes care of so much of the static dependencies. You just pull out the fun stuff and patch out the wounds. It does not require a lot of skill or knowledge, so a new project member can also be designated with a such task - not only the knowledgeable rock-star programmers that have the whole project in their head.
If you can assure me that in Clojure you can do the same in a safe manner than I'm all in. :) I'd appreciate any articles or stories explaining such scenarios in detail.
In Clojure, one doesn't need to rely on tooling to do refactoring, because things are isolated, loosely-coupled to begin with. There isn't much "unintended side effects" when one change Clojure code, because Clojure code consists of a bunch of functions working on immutable data structures.
The feel of programming in Clojure is very very different from what you are describing in a language such as Java. I have programmed Java since 1997, ever since I switched to Clojure, all those pains and fears of changing code has evaporated. I am not afraid of changing things in Clojure at all, because I know the effects of my changes are local.
Of course, Clojure toolings do include some convenient features such as "extract function", "rename symbol", etc, but these are just some niceties that save some typing and editing. They are not indispensable like those in Java.
Ok, so there are a lot less unintended side effects. I got that.
What about the intended ones - how do You approach them? Let's say You've written a ClojureScript webapp using the Angular framework and you need to migrate it to V2. AFAIK there were some breaking/conceptual changes which force significant rework. How do You make sure that after changing it everything works - even the most obscure option that only one user uses (as in the notorious example of Search Keyboard Shortcut in Outlook by Bill Gates)?
> You've written a ClojureScript webapp using the Angular framework
I know this is an example but Angular is terrible with Clojurescript. Clojure has an opinionated idea of how state should change over time and that opinion does not include two-way bound attributes. The reason the cljs community is virtually all React is because dom diffing makes the UI effectively a projection from state and that is epochal-time compatible.
> How do You make sure that after changing it everything works - even the most obscure option that only one user uses?
Depends how your app is designed. In the general case where, say, you're doing the equivalent replacing Angular with Ember you just have to do it and lean on tests/QA. I have just recently finished such a refactor and it's about as fun as you're implying.
With the refactor, I have a better plan going forward! We're now on the re-frame model. All state is now in the global atom (instead of mostly being in the atom but some being component-local), dataflow is unidirectional, and we're capturing all non-determinant actions as event params. Together, it means serializing the state atom and a sequence of events allows playback on the changed code. At least in theory. I wound up having to ship rewrite delayed features and haven't taken the time to build recording/playback support.
One would normally use a name space to group together code that deals with a foreign library. So, upgrading to a new version of the library is just a matter of 1. update the dependencies by changing the version number of the library in project.clj; 2. load the library; 3. changing the name space (a file) that deals with the library, and 4. changing the corresponding test name space (a file), and make sure all tests pass. That's it.
The Clojure coding process is REPL driven, so basically one tests every changes. Also, one codes in a bottom up fashion, starting from simple and small functions, and composing them into big functionalities. There' no big design up-front. It's an exploratory process, and it's fun.
> Another project starts up and wants to use the nice UI code. We designate it for a separate library. The first project must isolate and make everything generic. The requirement is that both projects must work on top of that library and the first one cannot loose any functionality
You are in luck! You used clojure and so 90% of your code is already made up of pure functions! And it's easy, with existing tooling, to move those functions to another namespace.
Of the remaining 10%, because you aren't running both UIs in the same process, the parts that will cause you problems are the parts that are Project A specific AND critical to the operation of the library. These are usually top level setup functions.
These functions, a half dozen for a complex UI, will need manual changes without much tool based refactoring assistance. Luckily because they tend to be setup style functions, existing tests and playing around at the repl will allow you to quickly confirm the refactoring was correct.
So in the end even with a complex refactoring like you described you only need to manually worry about maybe 100-200 loc in a 10kloc codebase. And it should take you a lot less time and effort than refactoring the code would if it was in Java.
I agree with others that less tooling is needed then usual for clojure, however, DEFINITELY check out cursive clojure. You can do exactly what you're asking for.
Right now I am working on a back-end API, and with the REPL and Intellij+Cursive I can't imagine a smoother experience.
To your point, by using clojure I can startup a debug repl, and launch the web server from within the REPL. At that point, I can
a) breakpoint at any line of code and see the state of the system.
b) manipulate my system in real-time and see changes in real-time without needing to recompile anything
c) if I see a function and I want to see the definition, or follow the call stack to figure out which lib this particular function came from, I can do that with a single keystroke via cursive.
Plus, you get all the features of inline docs, source code of function, etc (available via REPL or cursive).
My preferred approach with the dynamically typed languages is to have statically typed big picture (i.e., static module dependencies) with dynamic typing confined to module scope.
Another important thing is to not expose APIs. APIs suck. Use eDSLs instead - they give much more flexibility to a code. You won't have to modify the caller code when you change anything in the implementation. It's an order of magnitude more powerful encapsulation than anything you'll find in the OOP.
And, this way your code discoverability will be nearly on par with fully static languages, without the need in fancy heavyweight tools.
Can you explain in more detail or point at an example? How do you provide typing across modules? With type hints?
eDSLs instead of APIs - not changing client code... This sounds a little too good to be true. :) Can you provide an example. let's say for a UI form case?
> How do you provide typing across modules? With type hints?
By only exporting simple functions (which only take DSL commands and return DSL commands) out of the modules - it works well with Racket and some other Scheme implementations, for example. In Clojure you can simulate this with namespaces and an explicit "(require ...)" which can be easily followed by the static tools.
This way every name is either locally defined or coming from one of the modules you explicitly required, and with no higher order functions allowed in between modules you can track everything to its source.
> let's say for a UI form case?
That's the most classic example of this approach, used more widely than any other message DSL case.
An UI frontend is forming messages (like "(button-clicked 'Save)" for the low level approach, or, for higher level, like "(ui-do-save)").
A backend is receiving nothing but a stream of DSL commands from the frontend (i.e., no API calls in between, the stream can even be serialised as text), and execute them one by one, replying with another stream of DSL commands for the frontend (like "(go-to-dialogue 'Choose-File)", or "(hide-save-options)").
The more abstract your command stream is, the less changes need to be propagated either way.
Not quite. OO messages are flat (a method + a number of arguments), while a DSL messages can have a very complex semantics (even Turing-complete, if necessary).
What is common with the Kay view of OO (and different from all the modern OO languages) here is that messages are asynchronous, and this is a very important aspect.
If working with emacs, clj-refactor in CIDER can do renaming of symbols.
Of course, renaming symbols in Clojure is not such a big requirement as in Java IDE, because most of the things are isolated within a single function in Clojure. Consequently, renaming symbols is just a convenience feature, not a must-have feature for Clojure development.
If you mean keywords by those used in maps, like {:some-key 1 :another-key 2}, yes, they have project wide semantics. However, these are just data. Changing the name of a key in data is just a matter of global search/replacement.
A fundamental point in the philosophy of Clojure is the emphasis on raw data. This is actually the reason why we love the language, because it lets us work directly with data, instead of dealing with language related complexity, such as API, type, syntax, etc. The Clojure community only realizes this advantage a couple of years ago. This Rich Hickey's talk in JavaOne gives a flavor of this idea: https://www.youtube.com/watch?v=VSdnJDO-xdg
The tools exist for the low-level work of refactoring, i.e. emacs and clj-refactor. For larger structural changes, however, I've found core.typed has paid off. Obviously some people have a visceral reaction to type systems, but even if you don't use it at the outset, the ability to gradually add type information to existing code is great for paving the way for larger refactorings.
The workflow I have grown to love in clojure is an initial freewheeling, REPL-driven, dynamically typed phase in which you explore the problem you're working on, followed by a period of stabilisation that results in nicely factored, mostly-typed code. I personally think this can often give you the best of both worlds - quickly find the right design for a feature using clojure as a dynamic language, then robustly integrate it with the rest of your project using clojure more statically.
Speaking about my experience only: I do not refactor that often. See no point in it, Lisp is not Java after all. My code is usually well localised, with only languages being passed in between modules (instead of APIs). And modules are small.
You don't. See my long comment on the actual post. I've tried and I've tried and I've finally given up. The only thing I haven't really tried is "plan stuff so thoroughly in advance that you'll never have to refactor". Though this seems completely antithetical to Clojure's raison d' anyway, that I doubt I'll ever get there.
I think it was on google groups somewhere but I can't find it offhand, but one of the early big contributors of prominent clojure libraries has since completely quit clojure specifically for that reason.
I think it was on google groups somewhere but I can't find it offhand, but one of the early big contributors of prominent clojure libraries has since completely quit clojure specifically for that reason.
The only one I can think of, the complaint was the contribution/review process (I think it was Zach Tellman who contributed some work only to have Rich rewrite the entire thing out of the blue 6 months later). Unless you're thinking of a different case?
That doesn't sound right; it was something specifically to do with the type system, someone who was directly talking with Ambrose in depth, someone whose libraries I'd used, and who purely switched to Haskell. And someone influential enough that (rightly or wrongly) led me to follow suit (based on just a gut feeling and appeal-to-authority, and far less true investigation).
Ah, Ok, must be different. I didn't see the conversation you're referring to then.
EDIT: switched to haskell? Was it https://github.com/bitemyapp ? I didn't see the conversation you're referring to, but the comments he's made about clojure more recently made me ignore him; they weren't very constructive :(
Ah, indeed. Here's the conversation that convinced me: https://news.ycombinator.com/item?id=7051611 Given I was already leaning in that direction, it wasn't hard to be convinced.
Haskell was never really an option though because of libraries (and I'm not sold on Haskell anyway), so it led me to F# and Scala, both of which I "ideally" liked a lot. But lack of tooling and 3rd-party code, long compilation times, and obvious minority status on their respective VMs (e.g. nulls show up where the language wouldn't allow them) eventually drove me back to C# and Java. I don't like these languages quite as much, but all-told, when you throw in the ecosystem concerns, it seems like these are the languages that have the least friction against "just getting stuff done", and the ones I'm most productive in. For now at least.
S-expressions have changed the way I think, and it's really great. With paredit (or some other structural editor) and hiccup[1] I can make html so quickly out of pure functions that it's truly a joy. Using Clojure's datastructures, along with all the functions made to alter those datastructures gives such an immense feeling of power.
As disclaimer, I've been a Scala zealot for the last 3 years. On Clojure vs Scala, usually comparisons are kind of unfair I think. As somebody interested in FP, I tried Clojure first. And FP being all about immutable data-structures, pure functions operating on immutable data-structures and dealing with side-effects explicitly by means of abstractions such as the I/O Monad, I expected to see that in Clojure, seeing how people describe it as being very FP.
And it was like a big WTF for me seeing how for example everything in Clojure is a Var that can be changed. And eventually I got it, I mean Clojure is a LISP and it's meant to be compiled by a REPL and the unit of compilation is not a file and this is in LISP's tradition and so on and so forth. But it's still a WTF given its usual characterization. Even in Java classes are completely immutable (until you start manipulating bytecode of course) and you've got "final" declarations. You have no "final" in Clojure. And this is actually a pity because it (probably) means that those persistent data-structures can't be implemented in Clojure, as "final" on top of the JVM comes with a pretty cool contract in the Java Memory Model that you need. And then instead of Scala's implicit parameters, that are actually very explicit and type-checked by the compiler, in Clojure you have package-wide vars or refs that people manipulate as a poor man's dependency injection mechanism. And state changes are usually done by mutating atoms. Well that's cool, in Scala people also do that, but in Scala you also have a mature implementation of the I/O monad and monad transformers if you want it, or various FRP libraries. And I'm sure that Clojure has plenty of libraries, but overall I feel that in Scala the ecosystem of design patterns borrowed-from-Haskell is more mature.
Scala provides mutable variants of its data-structures in the standard library. But that's not extra rope to hang yourself, because in Clojure the use cases still exist and people end up using Java's mutable data-structures instead. In Scala the blend of OOP and FP can sound overwhelming, but it's actually the healthiest blend of FP + OOP to ever come up in a programming language. And in Clojure, just because you can't declare a class, that doesn't mean that OOP doesn't exist. It does and the blend is not healthy. This is why "reify" and "proxy" exist and are used quite frequently. And things like clojure.lang.ISeq are a Java OOP interface and not a protocol. Basically Scala is trying to make the best of OOP, whereas Clojure considers it a necessary evil of the host platform that it can't get rid of. Pick your poison.
Huh? Changing a var? That is very unidiomatic in Clojure and very strongly discouraged. You generally would (def ...) a var once and then it never changes. If you need mutable state, you use an atom, ref or similar, but redefining vars is quite rare in my experience.
This is why "reify" and "proxy" exist and are used quite frequently.
Not that frequently. I've only ever used reify for working with Om (and not touched since I switched to reagent). I think the last time I used proxy was 5 years ago. Of course, if you're directly interfacing with Java, you may need to use both, but for my own needs, almost all the Java I wanted to interface with already had Clojure wrappers (which I guess internally used reify/proxy and friends, but I did not have to know about this)
Most Clojure devs (myself included) are not building projects right at the REPL and indeed we do typically write normal code files, each file is a namespace. The changing of a Var is not a typical Clojure idiom at all, and I don't think I've ever seen it done in any production code or the big Clojure libraries.
> In Scala the blend of OOP and FP can sound overwhelming, but it's actually the healthiest blend of FP + OOP to ever come up in a programming language.
Scala is more functional because it encourages immutability more strongly and allows better abstractions like typeclasses, higher-kinded types while being roughly even on the modularity front with OCaml (there are a few things easier done in OCaml and a few things easier done in Scala) and beating the shit out of F#.
Not having a separate module and object system is another plus when compared to OCaml.
It unifies OO and FP better than OCaml and F#, because OO implemented in OCaml and F# feels like the designers tried to make a point about how much they dislike OO.
What's the first thing OCaml/F# devs tell others about OO in their langauge? To avoid it.
They tell people to avoid the things called objects in OCaml which essentially are just row-typed dynamic records and an inheritance system. Really it's the inheritance system people are taught to avoid... Same as anyone in OO is being taught now.
OTOH OCaml, as an ML, inherits an outstanding module story which is really the core of OO anyway. I'd happily say OCaml is a better OO language than Scala for that reason.
The module system is the object system fwiw. The things OCaml calls objects are just dynamic row-typed beasties which is why nobody recommends them (except to use as phantom parameters to get your own row typing).
Although I rather prefer the functional first approach of the ML language family and actually dislike how typeclasses and HKT are currently supported in Scala.
They look like workarounds based on programming patterns trying to make Scala look like Haskell, with proper support only expected for Dotty, if I understand them correctly.
HKTs aren't workarounds. They're fully supported. I suppose you have to use type lambdas to partially apply type constructors but that's not a big deal.
Implicits for typeclasses aren't really a workaround. Implicits parameters are an extremely powerful type-level programming tool. Typeclasses just require slightly more boilerplate than Haskell. And encoding recursion in the type system on things like HLists is rather straightforward and readable using implicits. In contrast, Ocaml and F# don't have the tools to express HLists or much of the type level stuff in Shapeless.
If you want to "redefine" a symbol in clojure you are better to create a new lexical scope via let with the new binding. But even then, why not just use a new symbol?
> I can refactor a huge codebase in my IDE. Good luck trying that with Clojure.
Cursive Clojure, a plugin for IntelliJ has excellent refactoring support, especially considering how dynamic Clojure is. If you've come from a Java background, this gives you a lot of the refactoring and 'Where Used' functionality that you might be used to.
Most discussions on lispy languages focus on the syntax. The unsaid assumption is that the only diff between a lisp and a non-lisp is the syntax. However, using lisp actually makes you think differently about programming. It is that which for me personally, is the selling point of a lisp. The syntax itself is merely expression, and is honestly simpler to grok than most other languages thanks to its consistency.
This post focuses on Clojure as a very pragmatic Lisp, but that, while great in itself, is not what makes Clojure so interesting. Clojure has proposed an interesting solution to the shared mutable state problem without trying to reduce it as much as possible like Erlang and without being pure functional -- and thus changing the whole concept of state and effects -- like Haskell (Clojure is imperative-functional). Clojure's solution is that all shared mutable state be transactional (or atomic, which is a special case of transactional). AFAIK, this is the first language to do that (again, without going down the pure-functional path). An imperative language that doesn't restrict global mutable state, but makes it transactional is a very interesting case study.
I wonder why the author left the thread-first and thread-last macros as a footnote.
This:
(reduce + (map (comp inc inc) (range 10)))
And this:
(->>
(range 10)
(map (comp inc inc))
(reduce +))
Are totally different from a readability standpoint. Yeah, there's some initial syntax to get used to, but in the long run I've noticed it to be the difference between immediately understanding what some code is doing and having my eyes glaze over.
For the OP - i've recently been programming in Scala, mostly spark jobs, nothing too complex, but damn its beautiful. I love how expressive the language is (certainly it can also be very complicated in large codebases, as you use more features, but I haven't encountered that yet). Heck just the simple 'scala' repl + JVM interop makes developing Java code so much easier. You seem to have tried Scala too - i'm wondering what you think of Clojure vs. Scala and what made you switch?
I've been a Scala developer for the last 3 years and I'm in the camp that first played with Clojure and then switched to Scala. I'm still playing with Clojure from time to time, but I like Scala more.
The only difference one should care about is static typing versus dynamic typing and that's it. Of course it's not a binary black and white classification when looking at languages, for example Scala is less static than Haskell and Clojure is less dynamic than Ruby. But overall that's the biggest difference and that changes the style, the mentality you have, how the code looks like.
As an example you are much more likely to see in Scala the Maybe/Option monad, whereas in Clojure people prefer to work with null (nil). Rich Hickey said it himself that Option[T] only makes sense in a static language. In general in Scala you're more likely to see things like monads and functors being modeled explicitly. It's more natural to do that in Scala because it's all a matter of describing functions with certain signatures. And you know, when working with combinations of flatMap and map, it's much easier if you have the compiler telling you where you've fucked up. Which is why many people do use things like Scalaz or Cats or Shapeless. On the other hand in Clojure it's much easier to build things like transducers, because there's no type-system to stay in your way.
Clojure people work with lots of DSLs built over macros. You can see that in how Clojure exposes those channels in core.async and compare that to how asynchronous processing of events is done in Scala (e.g. Future, Iteratees, scalaz-stream, Rx, Akka actors, Reactive Streams, shameless plug: https://github.com/monifu/monifu). Of course, Scala has its own macros system, but like in other static languages, the macros in Scala are only being used by those that really know what they are doing. But because of that, in both languages you get features exposed as libraries instead of being special language features. For example here's Scala's equivalent for C#'s await/async exposed as a library: https://github.com/scala/async
Which one is better is actually dependent on the problems you're trying to solve. The problems I'm working on have a very complex business logic and are a better mix for static typing. This is why people will never agree on which one is better, because it actually depends on context.
The strength of a type system is that there is one.
If everyone uses his/her own self-made pet of a typesystem most of the benefits are gone as soon as you are more than a one-man-hobbyist project.
Probably explains why most Lisps consist of one-man-hobbyist projects, though.
Type systems are by-definition only a conservative approximation of all programs.
The good thing is that 99.999999999999% of all programs never need to be written, and there are plenty of type systems which deal well with the last 0.00000000001.
> No, it does not.
Ok, that leaves the failure of Lisp outside of hobbyist mom's basements a mystery.
Please eliminate rudeness from your comments on HN. You've crossed the line into incivility several times in this thread. That's unacceptable here. https://news.ycombinator.com/newsguidelines.html
Please also don't post unsubstantive comments. That means avoiding programming language flamewars, since they're only nominally about programming and really about insults and disses.
So now it's possible to use a Scala macro in the same module it's defined? Good. But then documentation is too much behind, and this is most certainly not good at all.
Why is reading from left to right counter-intuitive? Don't we read f(x, y) from left to right as well? (f x y) has the same order, even the same amount of parentheses.
> The use of s-expressions and their resulting prefix notation is in many respects the secret sauce of Clojure. However, it requires you to read right-to-left3 and inside-out. This is counterintuitive to most of us, and takes some getting over. It is a very real barrier to entry for Lisps.
Author still doesn't get it. Lisp must, in fact, be read from the outside in, left to right --- in the same obvious direction in which it is evaluated. It is not very different from a nested f(x, y, ...) notation, except that the commas are gone, and the f ( exchange position to the ( f arrangement.
As a trivial example of why outside-in is wrong, is that if you follow that order, you might believe that (b c) is a function call in (defclass a () (b c)). The most important fact is that this is a class definition (in ANSI Common Lisp). The leftmost symbol in the outermost form, defclass, is the what McCarthy called the "main connective". It determines the entire meaning of the interior of the form and all of its subforms, so it behooves us to visit that first. When we recognize that, we then know that (b c) are slots, and not operator b applied to argument c.
The right-to-left part is completely bizarre. Why, as a matter of course, would you read the arguments of a form before the main connective symbol? If that, you want, languages for you, there are: Forth, PostScript.
Ha, yeah, maybe. I guess what I meant is this, say you have some OO/Scala-like code like this:
mydata.doA(blah).doB().doC(blah2).etc()
You can read it left-to-right: I've got some data, I do something to it, I then do something else to that result, etc. Maybe it's just that I've been using Java, and the like, for years but this feels simpler to me than s-expressions. At least initially.
With s-expressions I get something like (ignoring threading macros):
(etc (doC blah2 (doB (doA blah mydata))))
So when reading, from a data perspective, you kind of read outwards from the inner s-expression (which seems right-to-left to me). Or keep a kind of mental stack, as you read inwards (left-to-right). No?
As much as I like Clojure the lack of static typing always made refactoring a pain. The lack of it makes this a beauty I just can't appreciate and see.
Yeah, definitely. My bad. It's definitely not idiomatic.
I think was trying to avoid using the anonymous fn syntax, as I thought that was be more off-putting, and partial hadn't occurred to me at the time. In the end, I think I just confused the issue.
In retrospect, I should probably have used a totally different example, and avoided map and reduce as they are FP idioms. Live and learn, as they say.
- Great write up, very approachable for who may be on the fence about toying clojure.
- Re: Enterprise, while our enterprise footprint isn't ubiquitous yet, there are a couple of big name companies using clojure pretty heavily. Amazon, Walmart and Two Sigma amongst them. Also, because clojure sits on the JVM, we have a good interop story with legacy java codebases.
- Re: Refactoring, you should checkout intellij + cursive clojure, it's made refactoring codebases much easier for myself.
Re: Enterprise, I've seen a presentation about some of the work going on at Walmart. Would be great to hear more 'stories from the trenches' from larger scale developments. It would really help sell the language.
Re: Refactoring, yeah, I'm totally busted on that one, my wording was way too strong as I haven't tried either Cursive or clj-refactor. Plus in a way, it's an apples and pears comparison, between OO and FP refactoring.
For mobile a lot of folks are now using Clojurescript. There have been several highly rated discussions on here about the breakthroughs in the Clojurescript community for getting it to operate very quickly on mobile devices.
I can understand, but JS is not what it used to be. Things like React, OSX JavaScriptCore, Qt's native JS engine (which also allows you to build native apps for any desktop or mobile platform using Clojurescript), the use of Clojurescript server-side with Node, the state of affairs in JS are changing very rapidly, and Clojurescript is right there for leveraging all of it.
It's not any different than Clojure and ClojureCLR running in a "container of some sort". Clojure and ClojureCLR are libraries that add embedded support to the JVM and CLR to execute Clojure code and interop with the "native" environments there. JavaScriptCore is an embedded JavaScript runtime that provides essentially the same thing for JavaScript <-> Cocoa. You have full access to native Cocoa APIs. (Similar to Rhino or Nashorn if you are more familiar with the JVM.)
You use native widgets (e.g. UITableView), not web/dom widgets. This is a different thing than Cordova, et al, that essentially run a webview with some native functionality surfaced to the dom.
This is what React Native uses, and I thought I had seen some people experimenting with using ClojureScript and JavaScriptCore.
That's not entirely true. React Native does indeed run on the native device, not in a JS "container". The JS part is a bit of glue code. In the case of iOS for example, React Native uses the same exact UI toolkit that runs natively on the iPhone as if you have written it in Swift or Objective-C. Apple's JavaScriptCore exposes the native elements with a JS api, but they are not running inside JS. Only a minor part of the glue is interpreted as JS, and it's not usually the CPU-intensive stuff.
They are both beautiful in their own way. Haskell is more pure, certainly, and closer to mathematical beauty, definitely. But the extraordinary flexibility of a lisp, which Clojure does immensely well, makes it more powerful than any language I've experienced for quickly expressing or manipulating an idea. Any program written in Clojure could eventually be refactored into another language, but its strength as a dynamic lisp makes it possible to really get to the heart of a problem and work out its solution without anything else getting in the way.
As someone who has written only small chunks of lisp but at many different points in life, I do find the whole parens game to be a bit disorienting, but usually I can counter it by using newlines.
Haskell was created to be beautiful. Clojure was created to be practical, with plenty of ugly pragmatism thrown in. If there's an argument over beauty, it should be Haskell vs Lisp (vs Prolog, vs Forth, and probably vs assembly).
Bookwise, I thought the following were pretty good:
'Clojure in Action' by Amit Rathore and 'Clojure Programming' by Chas Emerick are both solid intro Clojure books. 'Web Development with Clojure' by Dmitri Sotnikov is also a great book, which obviously comes with more of a web-slant.
And I'd give a second thumbs up for 'Clojure for the Brave and True'.
As a person who doesn't even know what language this is written in, I don't think it is readable at all. For example, this "Select(i => i + 2)" piece looks totally strange for me. What does it do?
After learned Lisp, I feel there's too much syntax to learn for non-lisp languages. Every language introduces some idiosyncratic syntax. Life is too short, I just don't want to learn syntax that someone else invented to just be able to say "this is different".
This is almost close to showing a code snippet like this to someone trying to learn Haskell:
(it only prints the powers of 2, but it was enough to scare some people into calling Haskell “the Taliban version of ML”... http://www.theregister.co.uk/Print/2012/12/21/financial_soft...)...really, people, stop scaring learners away!