I am not presenting breakthrough material here. I wrote this because there is still a lot of confusion about concurrency and people looking for hot new solutions without understanding the basic tradeoffs.
You have not done much to clarify the confusion, though.
The tone of your blog post indicates that absolute truth and universal advice is to follow — and then it doesn't. What I got from your blog post is that one needs to consider various concurrency primitives and solutions (duh) and that you used Java locking, Java frameworks and Hadoop, and haven't found much use for Clojure's STM. Well, good for you, but I guess people expected more from a blog post titled "Concurrency, State, and Functional Programming".
There are two aspects worth mentioning:
1) It doesn't matter how fast you get the wrong answer, so your concurrency approach had better be correct. This is much easier to achieve with Clojure STM than with manual locking and mutable data structures.
2) Cost/benefit ratio: assuming each concurrency approach gives you some benefits (usually execution speed is what you're after), what is the cost? Cost meaning your effort to write the code, debug it, test it, maintain it, document it, cost for others to understand it later?
For what it's worth, I am getting very good mileage out of Clojure's STM. For scaling on SMP machines it gets me good results with minimal effort. I get correct code (#1) and the cost/benefit ratio is excellent (#2).
>. This is much easier to achieve with Clojure STM than with manual locking and mutable data structures.
For easy stuff yes. For hard stuff that is such a grandiose claim to make, if you could do it with any certainty you would certainly be published. Personally, I don't think it's any easier.
