A single erlang node runs on a dedicated OS process. It consists of one or more Erlang processes, which are essentially Carl Hewitts actors, which are very lightweight, don't share any memory, and hundreds and thousands of them can be multiplexed onto the handful of cores of a typical server. The idea being that if one erlang "process" dies, the whole system doesn't come to a halt. There's also no stray pointers, because all communication is done through deep copying messages, so there's no pointers referring to the process of another memories process.
Where it gets murky for me is how it interacts with databases. There's mnesia which... I don't understand. Is it embedded? I know it can run on top of stuff like LevelDB and RocksDb, and those engines don't like you opening the same database multiple times on the same OS process. But how do you stop that happening?
Also how do you test the whole thing? You could test individual processes I guess, but testing the interaction between them all seems fairly impossible.
> There's also no stray pointers, because all communication is done through deep copying messages, so there's no pointers referring to the process of another memories process.
Doesn't this bog down from all the context switching and IO latency from message passing? Or does erlang have some special way of doing it
It can bog down depending on your communication patterns. Context switching isn't too bad, it's 'green threads' so there's not the OS overhead of switch threads when you switch between Erlang processes; it's not zero overhead of course. Messaging passing latency is mostly driven by memory latency and lock acquisition; if you're in a big system, message queue locks tend to be uncontested as there's enough things going on that it's unusual for two processes to message the same third process at around the same time. But you can't really get away from that --- regardless of your language, if you've got multiple threads of execution working on the same data, you've got contention and that'll increase latency; even lockless algorithms have contention on the atomic values, and you have memory communication between processors.
All that said, BEAM isn't chosen because of its high performance, it's chosen because it enables a simple programming model for distributed systems while being fast enough. It's gotten faster over time too; of course, so has everything else.
It's all green threads, so you don't have real context switches and the messages don't have to be passed between (real) processes. Of course there are downsides to that kind of approach: good luck calling a C library function that you have to pass a callback into, anything else where you need precise control over your OS-level threading is difficult (and while BEAM has a good reputation, there's always the risk of e.g. priority inversions happening), and if the VM scheduler doesn't handle your workload well then you're limited to whatever control they give you.
> Where it gets murky for me is how it interacts with databases. There's mnesia which... I don't understand. Is it embedded?
Mnesia is a application that in my experience runs on some of your Erlang nodes; it provides APIs and also a supervision tree that owns the data. Mnesia is a layer on top of key/value stores; there's some out of the box stores it can use, or there's bindings for LevelDB and Roc0ksDB I guess. When you do a Mnesia read/write/etc, that becomes (more or less) a message sent to an mnesia process, so I'd imagine the LevelDB and RocksDB APIs are only called by a specific mnesia process and there should be no attempt to use the same database by multiple Erlang processes, so you're good from a concurrency point of view.
You can use Mnesia from remote Erlang nodes, it's just sending messages, so easy peasy. However, in my use, we would typically put state management services up on the same nodes storing the data in Mnesia, so we'd send an application message to a process running on the node, which would turn that into mnesia reads/writes as appropriate, and send back a response if appropriate.
If you wanted to use an external database like some sort of SQL, you'd need to find a client and then you could interface with that like in any other language. You'd almost certainly want to put some sort of intermediate connection pool together though, cause most databases aren't thrilled with tons of connections (although there was an article today about going to 1M, so maybe). Our needs fit well with key/value though, so I never looked into that.
> Also how do you test the whole thing? You could test individual processes I guess, but testing the interaction between them all seems fairly impossible.
Everybody has a test environment; some people even have a test environment that isn't production. :)
Testing distributed systems isn't simple. You can test the individual processes, but the interaction between all of them can give rise to emergent behavior which is difficult to test. However an important thing to consider is the importance of testing is related to the cost of failures; if you lower the cost of failures enough, comprehensive testing is much less important. Isolation of processes means that in many cases the rest of the system goes on when one piece fails; of course, if that once piece is in the critical path, that's not true. Hot loading means you can update code without losing established state, which can significantly reduce the update cycle time compared to a system where you need to drain users and restart servers or even longer cycles like in mobile development or even lengthier cycles of traditional boxed software. If you were writing Erlang for boxed software, you'd need to spend a lot more time on testing than in a messaging server.
A single erlang node runs on a dedicated OS process. It consists of one or more Erlang processes, which are essentially Carl Hewitts actors, which are very lightweight, don't share any memory, and hundreds and thousands of them can be multiplexed onto the handful of cores of a typical server. The idea being that if one erlang "process" dies, the whole system doesn't come to a halt. There's also no stray pointers, because all communication is done through deep copying messages, so there's no pointers referring to the process of another memories process.
Where it gets murky for me is how it interacts with databases. There's mnesia which... I don't understand. Is it embedded? I know it can run on top of stuff like LevelDB and RocksDb, and those engines don't like you opening the same database multiple times on the same OS process. But how do you stop that happening?
Also how do you test the whole thing? You could test individual processes I guess, but testing the interaction between them all seems fairly impossible.