Hacker News new | past | comments | ask | show | jobs | submit login
Clojure at a Newspaper (pitheringabout.com)
70 points by tim333 on Jan 2, 2015 | hide | past | favorite | 15 comments



I can add some more details here, because I'm still at the MailOnline :)

The CTO, Clifton Cunningham, did help by giving us the space to create new things. For example we also work on our own Javascript library for building the in-house CMS - https://github.com/MailOnline/milo

However, Clojure (we also use Node.js BTW) really does work for us. In the 11 months since Jon's blog post, the code has grown (I guess 30-40k LOC across several processes) but it also does much more too. I think Clojure helps because it gives more freedom to refactor/factor the code in the way the team want.

For example a decision could be made to create a microservice for a commonly used URL or API. Sure you can do that in other languages, but it just seems easier with functional languages, I think because the composed functions seem easier to lift and shift. I've no evidence for this, just gut feeling. I spent years doing Spring/Hibernate etc and really don't miss it.

We'll definitely continue to support Clojure and other functional languages.


With 30-40k LOC, could you please share the extent to which you feel protected when refactoring given that Clojure is dynamically typed? Some of my colleagues argue that often times a static compiler is the only thing keeping the ship upright in a large org with a big codebase, and unless your team is small and very smart, it would be foolish to use a dynamically typed language.

My experience can't help me much, but I sure want to try it.


Not to talk down Clojure, but it sounds like the real success here was being able to reinvent existing systems, and having a CTO willing to give you the space to do it.

After all, the majority of sites like this are getting data from <data store>, squeezing through <template> and delivering to the user's browser. The language you do that in will have a much smaller effect than, say, your caching strategy (particularly at a high volume site like MailOnline).


Conventional wisdom holds that rebuilding an old system is fraught with pitfalls. Executing is nontrivial and has zillions of decision-points where things could've gone radically differently.

Evaluating impact of tool choice requires experience and is intertwined with other factors. My experience tells me that Clojure's use is significant.


The CTO talks a bit about the rebuild at his talk on using Node:

http://youtu.be/HGH66JhUkis?t=8m59s

The old system was 155,000 lines of Java and similar. The team before him tried to rebuild using J2EE and XSLT and failed to get anything working in 8 months of trying. They then reorganised and rebuilt the system in about 4000 lines of Clojure:

http://youtu.be/HGH66JhUkis?t=16m3s


Sounds like us, 150k lines of Java/Escenic. Can't wait to move away from that hell hole.


I remember an old article on Java3D: rewriting a C++-based system in Java3D led to an order of magnitude improvement in performance! Of course, the lessons they learned from writing the C++ system could be cleanly integrated into a fresh Java3D implementation, so the comparison was not honest at all.


It would have been nice to see the comparison with independent implementations in different languages. It's too bad that they didn't persist with that idea.


I don't remember much about the node.js implementation.

Some early experiments with Ruby were beset by problems the VMs - Ruby MRI vs JRuby vs Rubinius etc ... We also had strange performance issues on JRuby, but not sure we ever really tracked it down.

In comparison getting Clojure running was easy. I felt so comfortable that I even tried emacs for a while (vim user) :P


Pure functional transformations over immutable, persistent data structures like Clojure's are an absolutely no-brainer for this type of apps where you get data from a database system > transform > display like HTML, JSON, etc.

I encourage anyone used to an object oriented approach of doing this to give Clojure a try. I used to do this for years in an MVC setting - it's a huge effort for no benefit whatsoever.


A follow-up which describes some of the technical pit-falls they encountered is also posted on his blog: http://www.pitheringabout.com/?p=1041


Great language, great community. Glad to see its adoption spreading.


Interesting. I wonder which library the author used for Elasticsearch, if any. For example, there is the official Java library (http://www.elasticsearch.org/guide/en/elasticsearch/client/j...) and several Clojure libraries (e.g. http://clojureelasticsearch.info/).


I haven't been in our Clojure code for over 6 months, but I'm pretty sure it's still https://github.com/clojurewerkz/elastisch


In answer to a comment on the article the author states that they're using Elastisch with the native driver (which provides 5X better performance than the RESTful driver).




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: