I just wanted to say that both this write-up and in the whole book you can see passion, and excitement for the topic. They say cliches like "work of love" but if anything that defines it I think.
It is the little humorous bits, the additional "Don't Drink Too Much Koolaid" parts, the visuals. It is a fantastic resource.
I think this post is a partially the response to "Why Cool Kids Don't Use Erlang" topic we discussed recently. There was a lot of "Too hard to get started" and I think Fred took the initiative and did something about it. That is great. Thank you Fred!
Thanks for the write up, especially the part on relx was enlightening. I've been toying with Erlang, and have three questions I have not been able to find good answers to. Maybe someone could elaborate?
1) How well does Erlang deal with multiple VMs on a single machine? Are there any upsides to using a single VM per machine? (So, run your own code along side a RabbitMQ instance for example, if that is even feasible.)
2) What is a good way to have run-time mutable configuration data? I was looking into Mnesia to store user / password combinations, but it seems a bit overkill. Ideally a solution that lets you add / remove config data from a running node on the fly.
3) How do you handle access control to the node, while preserving the distributed Erlang features? VPN all machines and then only listen on the private interface?
1) It deals with it well. There's no issue running one or more nodes on a single host or machine. It's been done that way before SMP was made nicer a few years ago. The distribution mechanism is based on a name@host scheme so as long as the total pair is unique to a cluster, things are usually fine.
Running one VM per machine has a few upsides: better resource handling (large binaries are shared on a heap rather than copied), copying within a VM is cheaper than serializing over the network, and so on. The downside is lower isolation.
For this reason, people will tend to bundle a bunch of services together if they relate to the same general logical block of applications. You should definitely be able to find a way to run your Erlang code around with RabbitMQ (though that depends on their compilation process and assumptions as well).
They'll keep them separate for operational purposes often: some code bases are less mature, require more ops, crash more often, and so on, or have more erratic behavior, and so isolation is welcome in these cases.
Erlang comes out of the box with a configuration mechanism, which you may or may not make mutable at your choice on a per-application basis.
3) That's one way to do it, yes. This one or equivalent ones (restricting network access, IP tables, etc.) are the most used ones as far as I know. Some mechanisms can also include running an SSH daemon on the node and only allowing people with the right credentials to connect to a node. There's one that ships with the Erlang/OTP distribution.
1. I've tended to see slightly better overall performance running multiple VMs (nodes)/server.
2. We use zookeeper and cache locally with an ETS wrapper. It's not-erlang but there are a lot of nice zookeeper tools and the ezk module for erlang is very solid.
3. we typically don't do vpn as if the vpn goes down it is a disaster. dual nics are ideal. i don't always configure that way due to limitations of the environment (non-VPC AWS, for instance). but what are you trying to control? access to the node even if you know the cookie?
Thanks for the pointers. Regarding 3), access control was probably not what I was thinking about. More like, how do I use distributed Erlang without allowing everybody access to the node (even without knowing the cookie) and how do I prevent a MITM to snoop on my Erlang traffic. [1] seems to have some decent info on in, I should have researched some more.
This is a really great project that could really make going from zero to "productive" with a new language so much quicker and more fun. I can't wait for Rust and Elixir to happen :)
+1 Exactly what was required. It's hard to find opinionated guides on the web for this sort of thing and can be sometimes overwhelming for someone just starting out. Great project!
Is there a way I can subscribe for updates via email? Thanks a ton!
Any time you need to do multiple things at once, erlang (or elixir) is often the best choice.
EG: If you have a website and want to serve more than one request at at time. If you have multiple people chatting or sending mail, or instant messaging, or voice communicating, erlang is a good choice.
If you have a search engine with many people searching, while spiders crawl the web, erlang is a good choice.
The only "downside" to erlang is new syntax and that its "slower" on a given node. But this last one isn't really a downside-- you can scale a process to 20 nodes with erlang a whole lot better than, say, node.js or java or go. None of those really support concurrency properly (of the ones who even try.)
This makes erlang unique.
I think many engineers try to tell themselves that the JVM or node.js or Go can be concurrent, but they are really just making excuses for the fact that the erlang syntax is a bit of a hurdle for getting started.
I'm fine with that- it makes a great hiring filter!
If someones learned erlang, you know something that makes them better than most of the rest of the programmers out there who are happy to believe rationalizations that let them get away with not learning erlang. (And if you're not thinking about concurrency you're even further behind.)
Completely different definitions of concurrency. For Clojure what ships in the box as concurrency is multi-thread, single process concurrency(aka built around shared memory). Erlang devs knew that would never scale beyond a single box so they didn't bother going after it. They built single-thread, multi-process concurrency as the core of their concurrency model(share nothing). This means by default Erlang concurrency primitives work beyond a single node, Clojure's don't.
> They built single-thread, multi-process concurrency as the core of their concurrency model(share nothing). This means by default Erlang concurrency primitives work beyond a single node, Clojure's don't.
Well actually Erlang is not single-threaded it is multi-scheduler and multi-process. CPU or OS based threads are used for scheduler and processes (possibly hundreds of thousands of them) run on all of them.
Moreover processes are very small memory wise (only a few K) and _most importantly_ have isolated heaps (yes, all while running in the same OS-level process.
What you are mentioning there is called "distribution". This is what allows you to send a message to another Erlang process, running on a different machine, different continent and have it look like Pid ! Msg which is exactly how you'd send a message to a local Erlang process as well.
> Well actually Erlang is not single-threaded it is multi-scheduler and multi-process.
I think the parent and you are talking about separate things.
Erlang the language is single threaded. Erlang the language has no concept of threads. That's what I think the parent was referring to, since he writes "concurrency model".
The BEAM Erlang VM, on the other hand, uses multiple schedulers and multiple threads, pretty much as you describe. Most people regard that as a characteristic of the VM _implementation_, not the concurrency _model_.
How so? Isn't it closer to multi-threaded with threads having isolated heaps? Can execute lots of spawn(...) statement and now there are multiple threads of execution running.
> Can execute lots of spawn(...) statement and now there are multiple threads of execution running.
You would end up with lots of processes running. Sure they are implemented differently than OS processes, but that is an implementation detail. If you had an OS that could keep up with Erlang's demand for processes you could implement Erlang processes as system processes and see no difference in behavior.
The point was more Erlang processes can talk to processes on another box fairly transparently. Clojure can not, their concurrency primitives are intra process only with no way to reach out to another box without developer effort. If your definition of concurrency is limited to a single process(and call Erlang processes threads) then Erlang and Clojure are about equivalent. If you drop the single process requirement(still calling Erlang processes threads and running multiple copies of the VM) from your definition then Erlang supports more types of concurrency than clojure does. It can support inter process concurrency.
Go is concurrent, and so is clojure (as mentioned in another response of yours).
I don't know what definition of concurrency you're running with, but all of those I heard of, they make it possible to have it there. Not necessarily at the same scale or through the same means, but they definitely make concurrency possible.
I've used it professionally as a trainer and course developer (teaching Erlang), in Real Time Bidding systems for advertisement, and I'm currently at Heroku, where we use it for the routing layer and part of the logging layer of the platform.
Personally? To just restrict it to professionally, I've worked a few places using Erlang. One for providing resource discovery to the rest of the internal systems, one for a financial matching engine and currently at Heroku for the routing layer and the logging layer(https://github.com/heroku/logplex).
Very cool! I think detailed examples like this are much more conducive to getting a feel for a language when you're just starting out than the typical "sift through the official docs then build a Hello World example." I'm going to set aside some time to work through this (and then go through the official docs.)
Kudos for the Simpson's references; that's one of my favorite episodes :) I look forward to more of these.
Hey this is really nice. I tried to get into Erlang from reading Armstrongs book (Programming Erlang) and just couldn't grok it. I think the later chapters of that get a bit more 'concrete' and start making real things but I lost interest by that point. Reading this has made me want to try again, though.
http://learnyousomeerlang.com/
I just wanted to say that both this write-up and in the whole book you can see passion, and excitement for the topic. They say cliches like "work of love" but if anything that defines it I think.
It is the little humorous bits, the additional "Don't Drink Too Much Koolaid" parts, the visuals. It is a fantastic resource.
I think this post is a partially the response to "Why Cool Kids Don't Use Erlang" topic we discussed recently. There was a lot of "Too hard to get started" and I think Fred took the initiative and did something about it. That is great. Thank you Fred!