I know it's been said by many, but I will repeat once again.
Elixir is a very cool and powerful language for building web apps.
Having built my first product (https://retrochat.xyz) in Elixir, I can wholeheartedly recommend it to anyone who is not happy with their current backend language(Ruby, Python, Js, etc).
Docs, functional programming, OTP, ecosystem(Phoenix, Ecto, Mix) are really top notch. In my opinion, you will be able to reach much higher productivity (especially long term) in Elixir than in Ruby, which is known for it's high level of productivity. Sounds like a fairy tale, right? Well, that fairy tale can become a reality with Elixir.
The only thing that is a bit of pain is a good release support, but that has been changing for better.
Releases with Elixir >= 1.9 are now quite good. Run mix release.init, write some configuration, then just mix release and you get a tarball ready to deploy.
We use a multistage Dockerfile to build the release and run it on a barebone alpine image, and it's never given me a single issue since I set this system up almost 2 years ago when Distillery was still the release library of choice.
And it's never crashed once, but that's the magic of OTP :-)
That's the approach that I take as well most of the time. Makes it pretty seamless to then take that container and run it in your cloud provider of choice. I put together a blog post outlining the use of Docker + Mix Releases for a Phoenix app https://akoutmos.com/post/multipart-docker-and-elixir-1.9-re... to help people get started.
The one thing that could really push it into "dream language" territory is a sound type system. The Gleam language recently popped up as an up and coming solution to that... It's ReasonML/Javascript like syntax makes it familiar and has a good interop story.
I'd be interested in hearing more about this. I'm keen on learning Elixir, but have fallen back to Rails at least 2 times when starting something fresh, because I'm very comfortable with Rails ops. It's simple enough for my case and rails console is just a blessing to dig around when something weird is going on.
The hardest thing for me to get was that Elixir/Phoenix is not Ruby/Rails. They are very different, while syntax is very similar, Elixir favours explicitness unlike Ruby. This leads to a few more lines of code than in Rails. But in a complicated cases, Elixir's code becomes shorter and much easier to dig in.
Simple case: Adding a new column to a table
Phoenix:
1. Migration
2. update schema
3. update changeset
Rails:
1. Migration
Complex case: Add a new column with validation only on update
Phoenix:
1. Migration
2. update schema
3. update changeset
Rails:
1. Migration
2. add validation with only on update
3. pray it won't break anything in other parts of app :/
The complex case sounds like a simple one, but in a big codebase adding validations could be troublesome for a fee reasons, but let's not dig into them. In Phoenix you can have multiple changeset definitions with different validations for each of them, which makes it much easier to control when and what data is validated.
It's just one of the examples, there are many others...
I'd say, if you are content with Rails, it's great then! No reason to change a technology stack, because some random person is saying it's worth it. If you found a good enough solution for your problems, stick to it :)
1. The stuff that never goes down (gets automatically restarted with last-known good state) and the amazing runtime in general. Not to mention it's much faster than Ruby or Python or PHP... In some cases it's faster than Javascript although JS and Erlang/Elixir are, on average, tied on performance.
2. A switch in philosophy and the way you troubleshoot your app: from magic/implicit to explicit. Many people cannot cope with this one because the force of habit is extremely hard to overcome. I for one enjoy knowing what exactly is going on in my app and exactly where but many others disagree. Again, it's a philosophical thing; you either are into it or you are not. There is also the objectively technological aspect: zero magic and implicitness mean much easier bug fighting. You no longer have to dig around in callbacks and installed on-start handlers.
Make no mistake: it's harder to start with an FP language. Nobody is disputing that. The benefits to be sown after the initial culture and brain shocks however are a lot.
> it's harder to start with an FP language. Nobody is disputing that.
I'll dispute it. For me, the functional paradigm clicked much easier than OOP and it's zillion abstrations and patterns. I'd argue that the best thing about functional programming is it's actually easier to reason about.
Eventually, yes. But if you were immersed in OOP/imperative languages for 14-15 years as I was (before learning Elixir) then it is really hard to muscle through the exercises and learn to think the FP way.
After that happened however, FP became much easier and much more pleasant to work with.
agreed. I'd been going steadily more fp in my style for years while working in c/c++, php, C#, scala,coffeescript etc. before finally going all in with elixir for the last few years. It makes many, but of course not all, things more straight forward compared to imperative programming.
> Not to mention it's much faster than Ruby or Python or PHP... In some cases it's faster than Javascript although JS and Erlang/Elixir are, on average, tied on performance.
By the metric of the average Phoenix request taking between 0.3ms and 10ms as opposed to 100% the same app in Rails never responding for less than 230ms -- an app that I rewrote to Elixir/Phoenix from scratch over the course of months.
You are correct in your specification though; I should have said Phoenix in particular and web development in general, not Elixir itself. The BEAM languages aren't excellent in raw performance, that much is true.
Also I am yet to be impressed by JS web apps. They are quick when you spin them off on your local machine or on a world-unknown website that only you and 3 buddies visit. But I've seen NodeJS apps crumble at 10 reqs/sec routinely. I'll grant you that this was 2 years ago though.
---
In short, I am not claiming to have made a scientific and thorough research in ideal conditions. I did work with PHP and Ruby for a long time though (and have been partially exposed to Python), and the difference in performance and reliability between them and Elixir is night and day. I've seen an Elixir app with a rudimentary 20-tables data schema endure 1000 reqs/sec on a RPi 3B.
I can only speak from my own experience but here is a data point (note that I do not use Elixir only, but also other technologies, in production).
For instance, I implemented a HTTP "business proxy" last year, which needs to be able to keep a large number of concurrent connections & proxy the call to a slow responding remote server.
I made multiple implementations (including Go, Node, Ruby with Roda, JRuby, Elixir, Crystal).
In this specific case I had, Elixir throughput & stability was by far the best. The gap (under load) between min & max was also the best (all this without tuning), and so was the CPU use.
I'd be extremely interested if you open sourced all those repositories. I thought of doing something very similar and your shared experience would be invaluable.
I know (you are not the first to ask!) but honestly, sorry, I won't and I will let anyone truly interested to do their own experiments :-)
This type of topic is too prone to endless debates on the validity of this and that, and the fact that anyone proficient in whichever stack I tested could definitely improve it (including me on the subpart of those stacks which I'm most proficient with!).
I don't want to lose any energy with this, and I have made my decision for that business case (in this case, a very happy one).
That's fair and I respect your lack of desire to spend a lot of time on minutiae. Sure.
That being said, you can always release the repos and just archive them right away -- thus disallowing PRs, issues and any public commentary in general. And whoever wants to dabble further can just fork and refine them.
It's your call of course. But this is what I would do I weren't interested in further changing the repositories.
I don't have any obligation here! I'm just sharing my own little experiment, which you are free to disregard.
While providing little details, it is actually providing value to some here.
In terms of measuring success & picking stacks, I have also a good experience (after 35 years of coding - I hate to mention that and this is no "call to authority", but I've seen a fair bit of stacks), and Elixir is a good investment at the moment for my use-cases.
I'll pile on here with my experience. I took a fairly complex Python/Django app and rewrote it into Elixir/Phoenix (and ripped out nearly all the complex caching along the way).
The Elixir/Phoenix app rewrite was a little more than 10x faster, much more consistent in response times, and needed far fewer resources to run.
Elixir and BEAM are clearly enormously better at some tasks but general performance is actually very similar between Elixir, Ruby, and Python if you use similarly performant frameworks.
Ok, but why did my app get 10 times faster? Why do so many other people report similar results? Perhaps factorial benchmarks don't translate well to the comparison of complex real world systems.
I think, when it comes to web apps specifically, there are a few advantages that make a practical difference. These seem to be the major ones:
- Compiled templates and more efficient string interpolation
- Improved parallelism and lighter weight processes. This leads to better handling of many requests in parallel (esp. while most are just waiting on IO), better db connection pooling, less blocking, etc.
> You rewrote it. The language you rewrote it in probably has less to do with it than rebuilding all at once what initially was built over time/organically.
That's a very unfair blanket statement.
There are no ways to reduce the inherent flaws of mutable imperative languages no matter how much times you might rewrite the apps in the same language. You definitely can reduce bug surface but not by much; shared data is much more prone to bugs and human mistakes than a system with a runtime that puts immutable data and message-passing front and center.
I'll give you that rewrites do improve state of any random app. But I think you are overestimating this factor with your pretty broad and somewhat dismissive statement.
There's no magic of course. The difference in real world apps that many people in this thread claim are in favour of Phoenix comes from the responsiveness and robustness of Erlang's OTP itself.
(I'll again reiterate that I rewrote a Rails app from scratch to Phoenix and thus accelerated it at least by a factor of 20x. I am just a working guy and have no vested interest in spreading misinformation.)
A lot of web frameworks crumble or introduce abhorrent lags after a certain threshold of pressure. Elixir/Phoenix in comparison almost don't change their average response time until much later.
Truthfully, after reviewing the TechEmpower repo for the last day and a half, I can only see their Fortunes benchmarks as being half-adequate. Most of their benchmarks are extremely unrealistic in terms of web apps usage. At this point I am not even sure what their goal is anymore.
It is definitely way too simplistic to compare two languages based on factorial (also, fwiw, the native annotation is not doing anything in your example unless the Enum module and its dependencies are natively compiled too).*
Elixir (Erlang rather) does have some "secret sauce" that makes it performant on part of the web framework space. For example, the literal pool used by the compiler+VM alongside iolists+`writev`-based operations make it so most template renderings in Phoenix allocate as little memory as possible. This means that any literal string in your template is allocated once on boot and not per-rendering, and building the output buffer of the template engine does not concatenate strings either (which would generate even more garbage).
I believe Ruby started to gain some of these benefits with the frozen-string literal annotations. So you can opt-in and no longer allocate static strings on every render. But afaik the template engine still performs a reasonable amount of string concatenation at the static/dynamic boundary. It would also likely be possible to eliminate it by implementing something akin iolists/writev. Maybe it has already been done, I haven't kept up.
So in a sense, everything is doable in any of them, but in Erlang this is the default way to do it, so it pushes towards an efficient approach to work with I/O from day 1. Case in point: I didn't invent any of this, I just learned what was there. But you won't see those differences unless you are working on an application that is rendering medium to large-sized templates (which are most apps returning HTML) and benchmarks don't tend to exercise that.
Other benefits from Erlang that may shine in the web space is the per-process garbage collection. It may reduce the variance on the latency and for short-lived requests, you may not perform garbage collection at all. All the VM does is to reclaim the space once the request is over.
At the same time, there are benefits in Ruby and Python you won't find in Elixir. For example, if you need an algorithm that relies heavily in mutability, then Ruby and Python will definitely have an edge. Discord recently had an example of where they made an algorithm much faster by moving part of it to Rust to leverage mutability. But benchmarks don't tend to exercise that either.
In fact, benchmarks may highlight non-ideal behaviour. For fully IO-based exercises, I got faster results by running an acceptor per thread that reads+writes as fast as possible, rather than multiplexing requests on all cores. But in practice, most technologies that can leverage multi-core will prefer to multiplex because that will be beneficial as soon as you do any subsequent I/O or CPU work.
TL;DR - sure Elixir can be 10x faster in some workloads, sure Python can be 10x faster in others. But of course, it would be incorrect to use any of these results to say Elixir or Python is 10x faster than the other.
*Fun fact: I am working on some VM trickery that makes things like Fib/Fac 3-4x faster but at the moment the trickery fails to show any benefit on any slightly more complex code. If they were to accept it, your factorial benchmark would show something much faster for Elixir, but in practice very misleading. :)
> Other benefits from Erlang that may shine in the web space is the per-process garbage collection. It may reduce the variance on the latency and for short-lived requests, you may not perform garbage collection at all.
This one was a big draw for me. It was a major reason I got interested in Elixir in the first place instead of going the Scala/Akka route.
It's already the standard way in Ruby, even in Rails. Jeremy Evans did the template optimization you're describing using frozen strings 3+ years ago in Erubi and it was adopted by Rails just a few months later.
There's no long STW GC phase during requests in modern CRuby either. It's nowhere near as elegant as BEAM's heap-per-process design but the GC issues seen in CRuby < 2.2 have been gone for almost 5 years now.
GitHub retired OOBGC last year. Pause times are just 1-2ms and total time spent in GC is 0.5% on for the main application I work on.
Are you talking about generating better bytecode based on the semantics of Elixir or a bytecode optimization pass?
> It's already the standard way in Ruby, even in Rails. Jeremy Evans did the template optimization you're describing using frozen strings 3+ years ago in Erubi and it was adopted by Rails just a few months later.
Yes, I was aware of the frozen strings optimization. Thanks for confirming! The part I am not fully aware though is in regards to the output buffer, which is a separate optimization. Is the output of a template being stored in an array that is writev-ed to the socket? Or is a template still rendered to a large string? Or maybe something in the middle?
If you don't mind me asking, what is OOBC? My Google-fu failed me. In any case, 0.5% is quite good, although that is also dependent on how much garbage the app generates. And as you know some libs/frameworks tend to be more mindful of that than others. :)
> Are you talking about generating better bytecode based on the semantics of Elixir or a bytecode optimization pass?
Do you mean the VM optimizations I was working on? I was mostly AOT compiling the bytecode of a function. That removes the VM dispatch overhead which further helps with CPU speculation. I see very good results on recursive functions but not much beyond that.
It allocates a single "buffer" string. IO.write on Ruby does support using writev but the fairly performance difference hasn't motivated anyone to use it for template rendering yet.
Ahhh sorry I meant Out Of Band GC.
> Do you mean the VM optimizations I was working on? I was mostly AOT compiling the bytecode of a function. That removes the VM dispatch overhead which further helps with CPU speculation. I see very good results on recursive functions but not much beyond that.
Sadly this is also why the CRuby 2.6 JIT isn't very effective either.
V8 removed their baseline JIT because it wasn't worth it. Firefox is considering doing the same. JSC has a more complex but effective baseline JIT. Implementing one for BEAM would be quite a lot of work compared to a minor performance improvement.
With frozen strings, it means "foo" and "bar" won't have to be instantiated on every render, but they are still copied when written to the buffer (unless the buffer uses memory references to frozen strings, akin to a rope).
Now imagine the output_buffer is an array, it means less copying, but support needs to be added upstream and other places to handle them. You would also need writev support for arrays (or at least convert them to an iovec or similar).
Does this make sense or am I missing something obvious?
Btw, sorry for all of the questions and thanks for the linked papers. Now I have something to read on the weekend. :)
CRuby makes a bunch of memcpy calls but compared to everything else, they're fast enough they don't seem to matter much.
I've run perf on CRuby running a bunch of different web app type tasks (even rendering huge JSON payloads) and it's never shown up. You can see an example here: https://github.com/k0kubun/railsbench#perf
Perf output for when running Roda looks similar but vm_exec_core and gc_sweep_step are even more of a bottleneck.
To be honest, I think you are underplaying the value of the sharing optimization (be it via ropes, array buffers, etc) given it can considerably reduce the memory usage (which may put less pressure on the GC).
This is because sharing reduces the memory usage in more than half. Without sharing (but with frozen strings), the memory allocation is `1 * static_size + 2 * dynamic_size`. I.e. the cost is each static part, which is copied, and twice the dynamic size, which is created and then copied (I am assuming that most of the dynamic content is allocated on the request life-cycle).
With sharing, the cost becomes `0 * static_size + 1 * dynamic_size`. Eliminating the static allocation is particularly important in loops. If you have something like:
<% for ... %>
pre
<%= dynamic %>
post
<% end %>
The factor of `n` applies to both static and dynamic, so you have `n * static_size + 2 * n * dynamic_size` when not sharing but only `n * dynamic_size` when sharing. So you can completely remove a linear component from the equation.
You could say that if you use an array or rope, there is still an allocation as the rope or array grows, but that is generally much smaller than the strings being concatenated. It can be preallocated too if you compute the amount of static and dynamic parts when the template is compiled.
So while memcpy is fast enough that it wouldn't show during profiling (except maybe by increased GC times), you should see measurable benefits in terms of memory usage.
Unfortunately I cannot prove this in practice in the context of Ruby, so I will gladly accept if you think I am overplaying the value of the optimization. :)
EDIT: taking a glance at the TruffleRuby rope paper, it seems templating with erb is around 3x faster once they moved to ropes.
Thanks for the discussion! I'll try implementing it and see if it's faster. It might be the case that with mutable strings in Ruby it's only faster on really big templates? I think pretty much everything required is in place and it's just a case of connecting stuff together with an option to use an array, except for implementing writev for CRuby which also shouldn't be too hard. It can even be an extension for now.
By every real production load we know. Techempower have not been representative of anything. The only reason elixir and phoenix is not higher is because their code is buggy and they discouraged the community to try to fix it.
Can you give us an example of a performance improvement which was rejected which didn't break the rules? There were performance improvements merged just days ago with input from Jose Valim.
i did not say it was rejected. I said the community lost the momentum to work on it. Mostly because it felt nearly impossible to get return on perf. When people had momentum to do it last time, we were never able to get perf numbers and to merge fixes before the official run. This basically killed most of the interest by long time members of the community to engage with the benchmark.
I am ofc not saying it is a good thing from the elixir community. Just that it is the sad state of the momentum to change it right now.
Q: Do you guys allow DB connection pools be changed? I know for a fact from my production experience that a small enough number for a DB connection pool in a Phoenix project can cripple its performance due to a lot of requests waiting for the pool.
DB pool size is very subjective much dependent on how long a connection is held without making a query. A recent PR for Phoenix significantly improved performance by _decreasing_ the pool size: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5153
I meant more like, people were using pools of 10 on very beefy servers and were surprised at bad performance. I agree with the PR author that the DB pool should be anywhere from one to double the CPU cores.
Thanks for the pointer. Currently reading the docs. Wonder if I can contribute to another language (not just Elixir).
Beam thread handling is vastly different than node, I'd be deeply shocked to see node handle as well at load on the same hardware as a similar configured elixir/erlang stack.
> The only thing that is a bit of pain is a good release support, but that has been changing for better.
The pain in Elixir releases I've seen has nearly always been self-inflicted. You can just run "mix phx.server" on the server and have basically an identical deployment experience as if you were running Rails, Django or similar.
Releases are an option that brings some power you don't get in other languages (particularly if you're using Distillery, which makes hot upgrades possible).
I wrote a guide on Elixir deployment that covers a way to use the new mix releases and deploy to Render in minutes. It also links to guides covering a number of other patterns, including the one I currently use for Alchemist Camp. It might be useful for others feeling some of the pain you did in the past!
I actually completely agree that Elixir developers aren't many. What I observed however is that people who want programmers that know FP and can apply it to real world problems where teams of web devs can be reduced from 20 to 3... get treated like coding monkeys and not given much respect and creative freedom.
You can't replace a pile of clueless juniors banging their heads at the latest Node fashion with several experienced craftsmen and treat them exactly the same. It just doesn't work -- people get discouraged and leave. I've had that happen to three acquaintances of mine (and myself, although I don't dare including myself because, you know, our ego gets in the way of properly gauging our own competence).
I've been rejected with the generic "poor cultural fit" excuse enough times on interviews to realise that most managers and shareholders simply want soldiers blindly obeying orders and not captains that can successfully make a team pull through a tough challenge on their own terms as long as schedules and business priorities are respected.
So, to me, the problem is also in the attitude of the shot-callers -- not only in the scarcity or quality of the Elixir devs.
<sarcasm>
That, and `no matching clause for foo/3` being 2019's version of the runtime "undefined is not a function" that breaks your app in production.
But then again, who does not write unit tests to get 100% coverage against 110% of possible user inputs, in case you made a typo somewhere. Right ? Right ?
</sarcasm>
Interestingly, coming to Elixir from another dynamically typed language (Python), I tend to do much fewer runtime/typing errors than ever before. I might be more experienced, or the tooling and compiler are good enough to catch the most common bugs.
I get a few "no matching clause" with my data ingestion tool on my Sentry log because external APIs are flaky and a cosmic ray flipped a bit so that I'm reading incorrect data. A GenServer somewhere crashes, restarts and the data comes back clean, without any user disruption at all.
As you might have guess, absolutely not my experience at all. But as explained otherwise, I'm in situation where "bad" data just keep coming in (and "bad" just mean "data that exercises a path were a runtime error will happen because of a typo that the compiler did not catch.) So "let it crash" means "let it crash again", and I don't see the obvious solution...
But again, I'm completely ready to hear that it's just my setup, or how we failed to build the app.
Just wanted to confirm that it's completely possible to shoot yourself in the foot, which I'm surprised hardly anyone mention when talking about elixir.
I think in this case having bad input is part of your business logic and should be treated as such. This is very easy in Elixir, specially since you can overload a function using pattern matching and so you can create a function (defined after all others) that matches anything. This is basically your error case.
"Let it crash" is commonly misunderstood. It doesn't mean that you shouldn't treat the known unknown cases of your business (like bad input), it means errors that are generally not known a prior (a database crash or network partition, for example) does not need to have a treatment since the Elixir/Erlang will simple crash and restart.
Things like K8s also brings a similar thing that Elixir brings to you, however having this in language is powerful since it allows ulyou to treat it better, if you want.
I’d say it’s not really different than other dynamic languages in that you can get those runtime errors. Initially I got lots of those but after getting a bit of experience I hardly get them anymore aside from initial kinks after refactoring.
Maybe try VS Code with the Elixir LS plugin. It runs dialyzer automatically and highlights errors which also catches a ton of those runtime type issues. It’s pretty nice, often more expressive than standard typed languages.
part of the Erlang Methodology is to write code that gracefully recovers from errors. it sucks for the user hitting the edge case until you patch it sure but the approach has it's merits and leaves the majority of your users running unimpaired.
Fair enough. Should have stopped at the sarcasm point. Sorry.
However, this is not my experience. When something bad happens in my situations, it's usually in response to a bad input that is going to be repeated a couple milliseconds later. So the app just crashes in loops.
I don't understand how the "let it crash" philosophy is supposed to happen when the only thing that can happen after a crash is... Another crash.
Maybe my case is just too strange.
I agree with your frustrations but I don't think the blame is on "let it crash".
As far as helping prevent bugs, let it crash won't help. It is not a replacement for a type system either. A type system for Elixir would definitely help you prevent bugs (but unfortunately implementing one is challenging).
Let it crash is a mental model and a technique to handle bugs and avoid undefined behaviour when all other attempts to prevent them (be it via static types, testing, code reviews, etc) failed. Imagine some data validation functionality had a flaw and that causes bad data to reach a core component, would you rather:
1. have the core component continue functioning with said bad input, which may have unknown repercussions?
2. have the core component crash and restart itself with a known state?
So let it crash can be useful to guarantee your system is not going to venture into undefined behaviour.
I wrote a small example of this in a blog post about assertive code (http://blog.plataformatec.com.br/2014/09/writing-assertive-c...). If you compare the two get_token implementations, you are 100% correct that in the second implementation, if you receive bad input, the code will crash over and over again. But I would say that I prefer the crash, so I can think about how I want to handle said input, rather than have an implementation that works by accident (and the accident may be a happy or an unhappy one). Would I prefer to have foreseen the bad input in the first case? Of course! And do type systems help avoid undefined behaviour? Definitely! But not all of them. It would be great if we could have both.
On a larger scale, especially when building complex systems, you are going to interface with client/server, database, apis, etc and you will run into unexpected scenarios. You likely can't prevent, model, or foresee all of them, so "let it crash" works as a default behaviour to handle them. It is also a legit strategy for things you know that may happen, but the best way to handle them is to start over anyway. It provides a mental model for you to reason when things go wrong at each individual component, instead of trying to reason how an exception may affect the whole system.
I am not sure if this helps, so questions and feedback is appreciated!
The "let it crash" is mostly meant to deal with data coming from 3rd party API data IMO. So if somebody's gateway throws HTTP 503 errors for an hour, your app will eventually catch up with it when the other side fixes it.
There's no code in the world that can automagically recover from unexpected and not-coded-for user input, though.
Very nice article, thanks. Reading with a slightly cooler head than yesterday, I still have strong disagreement with this.
I think it boils down to this:
> Here I'm classifying bohrbugs as repeatable, and heisenbugs as transient.
> If you have bohrbugs in your system's core features, they should usually be very easy to find before reaching production.
I don't know about "bohrbugs", but the vast majority of my bugs are "i-am-an-idiot-bugs", and boil down to "some combination of the handling one of the many possible inputs is going to trigger some code path where a function does not have the proper arity because I made a typo somewhere because I'm an idiot."
I claim that those are not "easy to find before reaching production" as a human being. (And I argue a large part of them are trivially found by a decent type static analyzer. Which, unfortunately, dialyzer was not last time I checked, 2 years ago.)
> By virtue of being repeatable, and often on a critical path, you should encounter them sooner or later, and fix them before shipping.
I disagree with that. My bugs happen on critical paths, but with non obvious shape of data. (I receive input from untrustworthy, aka "real-world", sources.)
I've spent enough time fighting bugs that were in the critical part of the code, were tested multiple times at the unit, integration and acceptance level, with different kind of simulators, but still broke on the first real-life usage because one function down a stack was not handling a nil or expected a string instead of a number, somewhere.
Encountering those issues "sooner" becomes a matter of "having enough imagination when writing automated tests".
I've heard property testing could be a tool to "automate" such imagination. But I'd rather have something telling me I just made a typo ;)
> So really, how efficient is restarting as a strategy?
> Well for repeatable bugs on core features, restarting is useless.
Amen.
> if the feature is a thing very important to a very small amount of users, restarting won't do much. If it's a side-feature used by everyone, but to a degree they don't care much about, then restarting or ignoring the failure altogether can work well.
It's hard to not read this as "Let it crash" being a good philosophy - provided it does not really matter, if your application works ;) (I know this is not the intended message, but still...)
I had great expectations about this section of the article:
> > I like statically typed language, and I restart my daemon after unhandled exception. What is Erlang gonna win me on fault tolerance
> This question was asked to me on a forum where I was discussing programming stuff and discussing the Erlang model. I copied it verbatim because it's a great example of a question a lot of people ask when they hear about restarting and Erlang's features.
The author follows with question (which is, basically, my opinion), with a very insightful explanation of supervision tree and OTP, but unfortunately, does not seem to talk about the "statically typed" part.
Which is a shame ! Message-passing, supervision and OTP are the great aspects of Erlang.
They're extremely powerful when used appropriately (which, in my experience, mean "sparingly, and as a plumbing-layer over purely functional code".)
My whole frustration with Elixir is that I want static typing "behind" the boundaries of GenServers.
Sure, I understand "GenServer.send" can not be typed, in general, and I'm ready to do some level of input validation at the boundaries, but beyond that point, please, If I tell you to add two apples, and I mispelled the name of one of them, don't even let me run, okay ?
I also know that there are efforts to do that (alpaca & co), but they're just too early-stages for my taste.
As said earlier, if you're solving the kind of problems where you never those kind of "i-am-an-idiot-bugs", or you're just not as idiots as I am, you may not find this problematic. I do.
But then, for dramatic purpose, the article ends with this gem:
> At some point, a team member got a very expensive credit card bill for the logging service we were using to aggregate exceptions. That's when we took a look at it and saw the horror on the leftmost side of the diagram: we were generating between 500,000 to 1,200,000 exceptions a day! Holy cow, that was a lot. But was it? If the issue was a heisenbug, and our system was seeing, say 100,000 requests a second, what were the odds of it happening? Something between 1/17000 and 1/7000. Somewhat very frequent, but because it had no impact on service, we didn't notice it until the bandwidth and storage bill came through.
Really ? Where those 500,000 to 1,200,000 exceptions not blocking anyone ? Not wasting someone's time ? Not causing improper or delayed results ?
Or, worst, just slowly making people lose faith into your software, or in software in general ? (Okay, I'm being slighly grandiloquant here, time to wrap up.)
Depends. I have a cluster serving about half a million clients that send binary sensor data every 1-3 minutes. Sometimes the hardware has bugs and send erroneous payloads while the majority just continue running. So instead of spending a lot of time figuring out all of the possible malformed inputs I just let the processes restart in those cases and otherwise resume.
There are certainly ways you can get run away reactions that eventually take down the otp tree regardless of your settings and I probably do more input validation than strictly called for my the erlang guide books but not spending all of my time worrying about those edge cases does have it's upside with out negatively impacting other clients.
it calculates running and time segment based average/high/low/tally values and drives alerts off of them. Processing is done in stages it's not particularly hard to change how one stage works as long as you aren't talking about major struct type changes or message format changes as those are a bit harder to rollout if not harder to implement.
Elixir, the language, the frameworks, and the community have been a blessing to my professional life.
If allowed me to learn the paradigm of functional programming and build systems quicker than ever before. Combined with the Phoenix framework, I prototyped a tool within a few days that would end up changing my life. I turned it into a SaaS business (called FeedbackPanda) with my partner, and it has been turning my life around for the better.
Thank you Jose for being such a key person in all of that without knowing it. Your insights and determination to leverage new tech combined with a proven software paradigm. Spot on!
Also, don’t forget... huge props to Chris McCord, the creator of the Phoenix Web Framework which is the main reason for me to completely come on board in the Elixir ecosystem.
I had the joy of taking part in a Phoenix workshop that Chris held here in Berlin at ElixirConf EU. It was insightful, and you can really tell Chris is in it with his heart and soul.
Can someone post/comment a more critical analysis of elixir if they can?
It's really hard to figure out how useful it is because people always build up hype when it's not always worth it. All I ever see is how wonderful and magical elixir is and how it's literally life changing. I want to believe but I've seen technologies with lots of hype that didn't live up to it. Plus elixir is different enough that I have a really hard time jumping into it to try it out.
----
I'll also add that recently I had to build a backend for a long term project I've been working on. I seriously looked at elixir/phoenix, but from my research it didn't seem like the pros outweighed the cons. Like there are some technologies that are so useful they're worth the lack of a robust ecosystem or the steep learning curve that may be necessary. And I am not sure Elixir meets that criteria.
There's already existing technologies like python/ruby for fast development, java/groovy for stability and groovy for modern features, and finally golang which is really good for concurrency and building web related microservices. Golang in particular is nice because it solves issues like python's slowness and it's slimmer than java in many ways. And these solutions are already pretty good. It's not like it's all that hard to write a backend. Like the productivity gains from js to typescript are huge because js can be bad enough that there's a lot of room for improvement. Whereas how much faster and MORE productive can someone really be in elixir?
And I am not even going to get into functional programming which is a whole other beast. It requires significantly different style of thinking, making a lot of my experience less valuable, I don't think it solves solutions for an oop system that's well designed, and often times the implementation can be bad if the right tools aren't there. From my limited experience, FP provides some really powerful features, but at the builtin cost of being more rigid when I don't need it to be.
Here are a few legitimate complaints about Elixir (and in some cases, its host VM, the BEAM):
* No mutation support means that for some CPU-heavy tasks, it's not as performant as languages that allow mutation. Some optimized algorithms that rely on mutation can't be expressed directly in Elixir or Erlang, and instead need to be linked in through a foreign function interface.
* It's not on the JVM so we don't get to piggyback on the thousands of dev-years of VM optimization and ecosystem around it.
* It isn't invented at, supported at, and marketed by a FAANG-type company. Money moves mountains and Ericsson doesn't spend as much on Erlang/BEAM as, say, Google spends on Go.
* Comparison across types is not a runtime exception, so you need to guard against crap like `nil > 1 == true` yourself.
* Erlang standard library, which is idiomatic to use from Elixir in the same way of using Java classes from Clojure, is a bit of a junk drawer of tools. There's good stuff, but you may have to dig for it.
* Elixir still hasn't nailed the balance between creating small, independent pieces of code ("applications" in Erlang-speak) and wrangling them so they can be used as dependencies from monorepos, multi-repos, etc. Umbrella projects were an attempt, I don't like them.
* There is an impedance mismatch between Erlang/Elixir's configuration system and 12 Factor® ENV-style configs. (Full disclosure, I wrote a library for this: https://github.com/appcues/config_smuggler)
* The built-in Elixir code formatter removes trailing commas from multiline lists. I mean, come on!
CPU-heavy tasks may not be constrained as much on mutation as on the host VM architecture. The threading system purposefully prevents a thread from hogging all resources. And since you have to run every computation in a thread (process), there really is some overhead. As I'm sure you know, the BEAM is built for concurrency, not for parallelizing computations.
Take Haskell for example, which is also immutable but doesn't require you to spin up processes to do computations. It handles CPU intensive tasks significantly better, but is a worse choice for highly concurrent low latency soft real-time apps because of the scheduler and garbage collector mechanisms.
Edit:
Btw thanks for bringing up that comparison across types issue. Never realized that. What's your solution to it? I'd write guards on functions that filter input based on types, then have a final fallback with no pattern matching that throws an error explicitly.
I don't have a general purpose solution to comparison across types -- it generally takes the form of wrapping whatever might have yielded a `nil` value in a `case` expression. Not a big deal, but in cases where the rest of the computation would be garbage if a `nil` crept in, I would prefer if I could just Let It Crash.
>* It isn't invented at, supported at, and marketed by a FAANG-type company. Money moves mountains and Ericsson doesn't spend as much on Erlang/BEAM as, say, Google spends on Go.
Indeed it wasn’t. In fact it was designed before any of the FAANG companies existed. And was specifically designed for systems that have downtimes measured in seconds per year.
These are fair points, but I would say that even though it's not the JVM, BEAM is an impressive piece of engineering with a significant amount of dev-years in it.
if I may do a bit of self-promotion, even though zig is not ready for prod (yet) I'm writing a nif system for zig that will make that whole process dead-easy.
I can give you my point of view as a seasoned PHP / Symfony dev with a light Elixir experience.
Pros :
- Elixir is fun. It's hard to describe why, but it is. Piping is additive, so are genservers and supervisors (which lead to anti-patterns but still).
- Phoenix is well thought out, documentation is good. Plugs are so simple.
- Websocket support is great, obviously (and LiveView is a gift).
- Ecosystem is not that big, but mostly quality and the automatic hex docs is a great feature.
- Concurrency is obviously good and, most of all, only there when you need it (looking at you, Node).
- Speed.
- Coming from PHP, having the ability to easily run tasks outside the current request is so freeing and eliminates the need of many third party components.
- Testing is good.
- Mix is great.
Cons:
- I don't care about static vs dynamic types but I do miss the type hinting of PHP 7. Typespecs are helpful but not the same (and Dializer reports are not fun).
- As a follow up it's hard to beat the IDE support of popular languages. (the Elixir plugin for Jetbrains' IDEs is good though).
- Not anybody will like functional programming I guess (it did kick my butt at first)
- Phoenix doesn't provide that much. If I compare to Symfony (and I guess RoR, Django etc) it's really noticeable. Authorization/Authentication, Cache, Mails etc. Also I'm not a fan of the templating (again, compared to Symfony's Twig).
- Also Phoenix speed of development seems slower (I guess the team is small and also currently a lot of work goes on LiveView)
- What I would give to have the cross-build capability of languages like Go, or the simple deployment story of PHP etc.
- Obviously a smaller ecosystem and less support online (StackOverflow is not useful, better luck on the Elixir forum)
- Speed (for some things).
- I do miss the return keyword.
- I do miss constants (modules attributes are not a replacement).
- None of your colleagues want to use it (some people just hate VMs).
I can take a stab at this. For context, my background is in Ruby, NodeJS, a bit of Golang and Java. I've been doing work with Elixir full time for the last 3 years.
I could probably write a novel about the things I like about Elixir, but here's what I'd caution about for someone looking at adopting it over another alternative.
- Elixir is not statically typed and probably won't be. If static types are a tool you value, Elixir's success typing and runtime assertions might feel inadequate.
- Pattern matching requires discipline to not create heinous coupling. Pattern matching is wonderfully convenient for control flow and helps a ton with readability and avoiding deep nesting in complex situations. It's so convenient, though, that it invites reaching through abstractions into data structures in ways that make code brittle. The language doesn't really provide any guards against this, so it requires care and policing to keep things manageable.
- Because OTP is so powerful, it's tempting to avoid industry standard tooling and "just do it in Elixir." This is great for moving fast, but has the risk of pushing problems down to the line to the point that bigger system rework might be necessary. Background processing is a good example here. In Elixir/OTP it's easy to do everything in memory, but once you start needing failover guarantees, or need split out worker processes from a web server for independent resource utilization, you may end up getting rid of the initial implementation and doing the work to move over to a setup similar to what you might using for Ruby or Node anyway.
Learning to be productive in Elixir isn't really a huge investment, even for someone with a strict OO background. It's pretty light conceptually compared to other FP languages, can be written in an imperative-ish style if needed, and has clear, simple patterns for code organization. For every place you would consider mutating, you tend to find primitives that make working with immutable data structures quite convenient.
I've been playing around with Elixir and rewrote a small Rails app in Phoenix to get used to it. In the end, the benefits are not big enough to make the switch
Pros:
- Less magic. It's a lot easier to figure out what's going on under the hood.
- Phoenix is an improved Rails. Ecto is more flexible, the generators make more sense and everything is better organized.
- FP leads to cleaner code. The difference to a neatly organized Rails (with POROs & service objects) is not big though.
- Tooling/Mix is excellent.
- Elixirforum is a great place full of helpful people.
Cons:
- Small ecosystem and lower productivity. I often had to spend 1-2 days to write a feature where with Rails I'd download a gem and it's done.
- Elixir gets hyped up due to BEAM, OTP, stability, fault tolerance... - however, apart from some very niche cases most of its features can be replicated with background queues, Kubernetes, etc.
- Speed & hosting costs. Elixir is a bit faster than Ruby and the footprint is smaller. But a powerful VPS + CDN is so cheap nowadays that it doesn't matter much at my scale.
- It's an absolute niche language. There aren't many tutorials, SO answers or solutions for small problems - you often have to figure it out on your own.
Overall I'd say that Elixir is fun to work with and ist well thought-through. At the end of the day I only care about shipping products though. Writing my admin backend in clean FP code is nice, but I rather download a gem like Administrate and be done with it - so I have more time to focus on my core features.
From a business perspective I don't see many cases for Elixir: A small shop is more productive with Rails/JS/Python due to the vast ecosystems - and a bigger company will have trouble finding enough devs.
You'll notice that people who praise Elixir compare it to minority web technologies (Ruby, Python) or ones that are documented for various issues (Javascript).
Absent from these comparisons are the elephants in the web world: the JVM and .net. The JVM in particular is superior in my experience to Elixir in all respects: statically typed languages, battle tested servers and libraries, abundant documentation and discussions, top of the line performance, excellent tooling allowing rapid refactoring and evolutivity, etc...
critical idk. But i can answer your points against.
What if you did not have to choose ? It gives you all of that. There is no "steep learning curve". I had a sysadmin that never did anything functional pushing code to prod in less than a week. This is not a single case. This is the widely reported result across companies switching to elixir.
Regarding rigidity: quite the contrary. In my experience elixir and its take on FP are faaaaar less rigid than an oop one.
I am not going to bad talk any of the languages you offered. I will simply say that i disagree with basically every single one of the characteristics you offer for them.
Regarding how much productive... Well the most extreme example is whatsapp doing it all with 8 backend engineers. All of whatsapp. Recently RocketMortgage had equivalent results. In the case of rocketmortgage, it was wihtout any prior experience in the ecosystem. This was for Erlang, but all the companies using elixir report equivalent results. PagerDuty and Bleacher Report are probably the most visible one.
So in the end... do what work for you ? I do think you are missing the point a bit with your argument, which is what i will fight. But if other languages work for you, great for you :)
Wait, what? Whatsapp had 25 backend engineers in 2016, and they use almost exclusively Erlang, not Elixir. They’ve said many times that the Erlang platform has allowed them to do a lot with a small team, but let’s not turn their story into an urban legend...
no i disagree with go being good at concurrency and being good for web stuff for the go example. It is one of the worse experience i know. It is like trying to use C to do it.
It is a decent language, but it's development sucks. Maybe I'm missing something. I use it for Jenkins pipelines and it does not even has an editor with code completion. I must run it through an interpreter to find simple syntax errors, like a missing parenthesis. Looks like I'm developing in the 80's.
The Groovy compiler feels like a half-baked product, just as the language. It cannot decide if it wants to be a statically typed language or a dynamic one.
It happens all too often that I have to manually cast stuff because the compiler cannot deduct the type of an expression. Seriously, it feels like writing Java pre-1.5 (before Generics).
At the same time, your code might just compile, and you'd get a runtime error because you forgot to rename an attribute that you referenced, and due to some dynamic wizardry Groovy tries to retrieve it anyway.
It is the very opposite of the Erlang universe, where the more time I spent the more impressed I got. So really no comparison between the two, forget Groovy if you have more than a toy project. Java on the other hand... will soon have it's fibers thanks to Project Loom...
> your code might just compile, and you'd get a runtime error because you forgot to rename an attribute
The eclipse plugin has a pretty good approach in that it underlines things that cannot be deterministically type checked (that is: it can't prove they are wrong, but it can't prove they are right). It does a fair bit of type inference, so once you get used to that it's reasonably low amount of pain that comes from this type of problem.
> I use it for Jenkins pipelines and it does not even has an editor with code completion. I must run it through an interpreter to find simple syntax errors, like a missing parenthesis
It's sad that so many people form their perception of Groovy from what they encounter inside Jenkins.
I do full application development with it inside Eclipse and it is fantastic. None of the above applies. It's basically the same as developing in Java just 30% of the code size.
One piece of advice: go for Erlang, not Elixir. The syntax is much nicer, and you get a really mature environment. Also, 90% of Elixir is Erlang anyway.
I don't know too much about Elixir, but for implementing web services just use the tech you know the best - these days you should be able to scale almost indefinitely regardless of the stack you choose.
But if you have a problem that resembles a telephone exchange (eg. no UI, lots of concurrent IO intensive - but not CPU intensive - tasks that have soft-realtime requirements) give Erlang a chance. Even if you end up using some other technology it will really influence your professional development in a good way.
Elixir is much better in the respect that it makes it easy to do the right thing. Code organization, documentation, https, unit testing, even deployment, live reload when web developing, all work out of the box in elixir/phoenix in ways that aren't as convenient or pretty in erlang. Elixir in many ways adds cheddar to the broccoli to get the kids to eat it.
As a C# developer I've learned the basics of Rust/Elixir/Go and found that while they're all nice languages, my knowledge of the .NET platform is so deep that for kind of applications I develop (APIs mostly), the tradeoffs of using something new isn't worth it. I haven't come across a problem I couldn't solve with C#. I don't work at Google/FB/Apple, so I don't really deal with scalability issues. F# is interesting, but honestly it's so poorly supported by MS and the community that unless you work at Jet.com it's not worth it.
Don't get me wrong, I love learning new languages/tools, but the cost of learning them deeply is a price that's hard to pay for when you have to get things done at an ever-increasing pace.
I'd love to give you a challenge: Write a live-updating web dashboard -for anything- in Elixir with Phoenix LiveView. Then think about how you would do that in C#. Sure you can do it, but you can't do it as fast, and what you come out with in 5 weeks as an expert in C# (which should take you 3 days - including beautiful html documentation and all the unit tests - if you're an expert elixir programmer, or, maybe a week and a half if you're a noob) will not have the uptime.
I will guarantee you that an expert C# developer could write the same system in same or less amount of time than Elixir with Blazor or even SignalR. C# ecosystem is so big and robust at this time that almost all big problems have already been solved.
I built a personal project on Elixir, Phoenix, and Postgresql(currently on hiatus).
It have a somewhat steep learning curve with the ORM Ecto. Debugging Ecto, changeset, etc.. is not fun. Dumping whatever Ecto returns can be a puzzle because by default other than the main tables are not preloaded and it loaded with meta data. Also how you deal with Ecto and forms in phoenix is not clear, lots of gotchas. The common thing is that you have a form template that update/edit and new/create share. The problem here is if you don't know what you're doing one of your template will expect a changeset and the other will expect a list. I forgot which one gotta look into my code but it was an annoying gotcha.
The lack of things baked into Phoenix makes it a hunt to find packages/libraries to add features to it.
Examples are Laravel having rate limiting router built in and signed links built-in. I don't believe the html form is as good as other frameworks; can be at time confusing. I think while Phoenix may not be aiming to be full features as Laravel, I believe security measure such as rate limiter and signed links should be built in.
Deploying is also a learning curve. You need to compile Elixir in production server and then the folder layouts is different than development. So hunting for the root web folder of your application is a thing I did and learning about the layout of production.
Having said all that, I enjoy coding in Elixir and using Phoenix. It is fun getting better at it. While there aren't many thing baked in like Laravel, it made me more educated in the things I've taken for granted. I re-did the authentication system 3 times. I tried jwt, did more research and realize when to use session and when to use them.
I enjoy coding in Elixir more than PHP. I see myself building my side web projects in Elixir in the foreseeable future. I do recognize that it's a smaller community with less contributors so I hope this come off as a constructive criticism of something I love and know well enough.
I am so appreciative of his accomplishments. Knowing the history of ruby and functional languages, figured early on he found the right solution to the right problem and the impact of this language has been huge very quickly
How does one get motivated enough to say - you know what, I'm going to create a new programming language? How do they keep it going when they face hurdles? How long did it take to develop this?
I'm struggling to start any in-depth project. I can do stuff - make a small app - here and there. But the thought of starting something is so daunting and intimidating. The irony is that I built a small but working language as part of the compilers course in college. I feel like I know so much more know but can't bring myself to develop something that reflects that.
I personally started the whole thing for learning purposes. I don't have a computer science background, so it was exciting to learn all of the different concepts. Most of it was initially driven by curiosity and challenge.
But it is also worth saying my first attempt (at Elixir) sucked and it demotivated me to work on the project for 6 to 8 months. It was only later I learned/figured out some of the answers or possible solutions to the problems I was having and that motivated me to work on it again. Once I asked my partners to let me work part-time on the language, I prepared myself it would have to be a long term effort (at least 2 years). Then people started using it and that in itself kept me going (and it has been like that for almost 9 years now).
In all fairness, I also do struggle with motivation when starting hobby projects, even more larger ones, and especially if most of the challenges have been addressed. Also what keeps me motivated changed throughout the years, so I try to be mindful of that.
It is 100% a-ok that you don't want to do such a large project. It would also be a-ok if you wanted to. I've observed that for some people, obsessing over a software project is as natural as breathing. Not me though, I have completely stopped doing side projects and exchange code for cash.
As a beginner, who has never touched Elixir, where should I start? My issue is tutorials NEVER have production quality code. How do know what my code structure looks like? How does linking works? If I/m using database, how can I use connection pooling. Stuff like that. I can look up the basics, but I have always struggled to learn how use X language in production. But I guess this is can applied to any programming language.
After tinkering around in IEX and doing the Getting Started Guide, I'd do Dave Thomas's course (https://codestool.coding-gnome.com/courses/elixir-for-progra...) and read Elixir in Action by Sasa Juric. Dave Thomas's course will get you through the basics of the language and the important concepts step by step. Elixir in Action will go further into the concurrency model that makes Elixir unique.
It’s a shame elixir never really seemed to take off. I’ve been a huge fan of the language and recent Phoenix framework for three years or so. However, it seems like specializing in this Lang would be more of a hinderance than asset. Great for personal projects and a joy to work with, but for some reason a lions share of developers seem to be scared of functional programming.
I am using elixir for a very important part of a project where I work. The project leader who is giving up their repsonsibility over a half-baked python/django effort that is currently in prod asked me if it's risky because we're increasing our exposure by using a product that is not battle tested. Then i told him that our at work documentation (slab) and our soon-to-be site reliability early warning system (pagerduty) oh yeah and our corporate card (divvy) all are or Elixir/Phoenix so we already have exposure to the risks in adopting it.
The great thing that José really inspires is his energy. The moment while I'm learning Elixir slower than he's adding things to it, is the realization. Then I occasionally have a creepy habit where I like to stalk his git commits, to see what he's doing.
I was about 2 years into professional development when I started getting into Elixir. Recursion was a week or two of "err what?" then I got it. Functional composition is way easier to reason about than OOP design patterns (and should be since OOP design patterns are just over-complicated Category Theory).
The tooling gets out of the way, so a newbie is less likely to run into roadblocks. IEX is the best REPL I've worked with. Phoenix is much simpler to reason about than similar frameworks like Rails. And the docs are clean and consistent (+ dark mode). It was really easy to navigate through the docs, try things out in IEX, and get things working. It's just modules and functions. There's no classes, base-classes, abstract-classes, inheritance, instantiations and what-not. Just modules and functions. I doubt there's a better way to learn back-end development (DBs, Queues, Concurrency, PubSub, Distributed Systems, etc.) than Elixir.
A buddy of mine has been getting into Elixir recently while going to school for CS. The school's curriculum is mostly C#, Java, PHP, and what-not. He says Elixir demotivates him from some of the homework because everything in Elixir would be way easier.
I write CRUD apps for a living , how is Elixir gonna make it easier for me? It can't and it won't. Not saying it's bad or anything, just has nothing for me personally.
I also write CRUD apps for a living. Not every project needs Elixir, but here's several reasons why I might choose Phoenix for a project:
1. Phoenix's channels make communication between a frontend and backend pretty effortless. Create a channel, publish a message, and anyone listening to the channel picks it up. Communicating updates from the server to the client is simple. Communicating between clients is equally simple.
2. Elixir has great support for spawning and managing background processes. My go-to is Rails, and if I want to manage background work I have to use something like Sidekiq and Redis. It works, but Elixir is much nicer. I've written systems that need to manage heavy amounts of background work and I dream of rewriting them in Elixir.
I wish that more people would bring up points like this instead of just insisting that Elixir and Phoenix are "scalable". Both of those points sound appealing.
It's like, okay, so there's the functional programming, the similarities to Ruby, the BEAM VM, the concurrency.... but what does it do for me? They're not exactly selling points on their own because other languages can compete with those things.
As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.
You make an app using Ruby on Rails, keep developing it, and it keeps growing. You monitor your traffic and it gets bigger, now you add more servers to handle the traffic (it costs you $20/server, now you need 4). A couple more developments, memory usage starts to grow also.. Now, you need to increase memory of your server.. but now you also need Websockets, so you add both and need to add more resources to handle the scale, and the cycle continues.
When you use Elixir/Erlang and it’s BEAM VM, it’s highly efficient and scalable out of the box, so you won’t need to do “much” of the above. A simplistic explanation but shows the point.
OK, so in real life maybe the costs can actually be an order of magnitude higher than this.
But for $60/$80 a month? Is like no money.
That is less than one hour of time for even a cheap developer including taxes and other overhead, or okay call it two hours a month of time if you want to be generous. So if Elixir instead of Rails (or, of course worse, _switching_ to Elixir from current Rails) takes more than, say, 24 hours of developer time in a year... you didn't save money.
OK I'm not working on Google scale, but for the vast majority of companies I don't think this actually matters that much. If it's all about budget than I would hire PHP devs instead of Elixir cos cheaper! No it doesn't make sense sorry. For the small amount of companies where this actually matters you'd still need to convince me how java/.net/Go can't get it right.
ok maybe this make a difference. I can train a junior on elixir and have them writing code that I am confident won't break the system in prod within three months. I can't do that in any of the other languages, I would have to buy a senior.
PHP is trivial. Understanding it and shipping code can be done by a 9 year old. Rails is trivial. What is so trivial about Elixir? More concepts, caring way more about performance and concurrency (things a normal dev has to deal with very very rarely). Just don't see the upside
It's understandable. Elixir is one of those languages you won't really appreciate until you've experienced the problems that it addresses.
You're getting down voted mostly because of your tone (IMO) but the appeal if Elixir is just a reality of experience.
When I found it, I was 15 years in my career and when I dove in, learned it and saw how it addressed so many short term and long term issues that appear in application development...the balance was the closest thing I'd seen to perfect.
But if I hadn't experienced the pain, I would never have appreciated it all the little details.
It would be nice if you elaborated on what those problems are and how elixir solves them better than existing alternatives.
All I ever see is people exclaiming how awesome it is and then the responses turn out to be underwhelming for various reasons. Like hot reloading isn't that special or unique anymore.
But that's an interesting talk. I didn't have time to go through the whole thing but the core of it seems to be the pros of erlang like BEAM's high concurrency advantages and uniformity of tools running on top of otp.
Hot reloading is very unique though (but not necessary for most use cases). There's no other stack outside of the BEAM that has an equivalent. A lot of what is offered by the BEAM isn't even possible in other stacks because it requires a collective of language decisions working together.
Doing this for 7 years, have yet to seen a problem throwing more servers/caching/database locks can't handle. I'm not saying those problems don't exist, I'm just saying it's not that common. Agree about the tone, was having a rough morning (plus: why does every Rails thread turn into a "you should really try Elixir!" , yeah we don't like our stack to be constantly shat on by Elixir guys)
Of course you can throw more servers/caching/database locks, but the advantage of using Elixir & Phoenix is that you can do a lot more with less.
Think how much the code would be simpler and easier to reason about without caching.
Or reducing the numbers of servers you need from 150 to 2 and with better performance [1]
> yeah we don't like our stack to be constantly shat on by Elixir guys)
Sorry to be blunt but this is nonsense, languages and stacks are just tools.
When I was working on banking systems the idea of using dynamic typed language would have been insane. Now that I’m in a two man’s shop working on low traffic web app I’m happily using Rails for the CRUD work. And if tomorrow I were to work on a public high traffic real-time app like Discord I would do it in Elixir. There is no such thing as your stack.
Usually there's the 1-2 tools you're most comfortable with, unless you're a true generalist. Most people can't be incredible in Rails/Django/Spring/Node/.NET at the same time, you have to pick 1-2 things you excel at. You can be efficient in 5 tools at the same time, but not an expert (for most people. some people are great at everything they put their hands on).
You seem to have a bunch of experience with Phoenix, could you suggest some tutorials that will be helpful for people who start from scratch learning elixr and this type of development.
Coming from a php / node.js background it seems very overwhelming
CRUD is trivial in general, and pretty easy to implement even with the Node.js built-in HTTP library. A more complex API is probably where Rails or Phoenix start to make sense, but there's no reason why an HTTP library and a database aren't enough for basic CRUD. Someone who doesn't see CRUD as fundamentally trivial probably hasn't been programming very long.
I wouldn't say Elixir has more concepts, it is rather different concepts.
You also don't have to worry about performance and concurrency. I would actually argue that you have to worry less about performance and concurrency because you can go further without having to resort to cache layers, tuning, etc.
But ultimately, I don't disagree with you. If you are building CRUD apps, then any technology is likely fine. Perhaps there is a chance Phoenix will start to take a bigger pie of the "CRUD apps space" with LiveView (https://github.com/phoenixframework/phoenix_live_view). If you could produce CRUD apps at the pace you do today, but making them interactive and realtime is orders of magnitude simpler, would that interest you?
Well yes of course they do. Any modern web dev stack does concurrency out of the box. In Rails you use a web server like Unicorn if multi process is good enough or puma if you want mutli threads as well. PHP has apache modules that do the same thing.
considering their comment about it being difficult with rails, I think they are likely talking about concurrency within a single request as opposed to what I think you are talking about of concurrently handling requests.
Again, that's just very rarely needed. The server is concurrent and if you have very heavy stuff you use background jobs. This doesn't only work for Rails, this is what java, .net, php etc etc are all doing and it works. I'll be interested intellectually to see what Elixir can do but I just don't see how it solves anything that isn't solved already.
I'm not preaching the elixir word here either. I've never used it for anything real and just dabbled a bit as a toy. I'm just trying to help get this conversation going because I'm also interested. My experience is in java, node, and python primarily which covers a good bit of ground as far as the different concurrency approaches.
Anyhow, your characterization there I don't think is quite right. For one Java does not typically use a background job set up, because it is a first-class multithreaded runtime so it seems more likely prepared to use a threadpool for some concurrent processing before it say tosses things out to a task queue. Similarly as most java web servers are on a thread pool model for concurrency as opposed to async IO, you'll also have true in request concurrency for things like making a few calls to the DB and then collecting them. This thread pool is not well abstracted typically, and the mutable object model leads you to writing classically tricky multithreaded code with data races and the like.
For something like node where you are not really likely to be using threads, you are pretty hard tied to async IO as your method of concurrency. So you have a really easy job if all you're doing is async calls to say a DB with no need for data processing. Here once you need real computation you will have essentially no option other than using a task queue.
Then when it comes to python you have essentially the worst of all worlds. it's not easy to do concurrent IO or computation without some form of background processing.
I believe that when it comes to the elixir runtime the idea is that they have removed async IO from the mix and instead put up a concurrency API that is as simple to manage as async IO in something like node. Under the covers is all delegating out simple blocking code to a thread/process pool, but the immutable data model and API makes it so that data is not shared across the pool, trying to keep the classic concurrent mistakes hard to make.
Nothing that can be done in elixir seem to be impossible or even astoundingly hard in some other technology, but the unified API for all forms of concurrency seem like a nice way to make it a runtime that can handle many different types of workloads in a simple and cohesive way. That simple way also seeming to not need you to bring in things like a redis task queue where you generally would need to use it or some other external solution.
"Java does not typically use a background job set up, because it is a first-class multithreaded runtime " : Is there really no equivalent of Sidekiq in java world, or that's just rarely used? I would assume a separate process would need to run over the jobs (so can be scaled easily) so then you need to write the jobs to some shared memory like redis. Is this rarely used? https://docs.spring.io/spring-integration/reference/html/red...
The advantage of Java and Erlang the author was referring to is that you don't need extra technology. With sidekiq, you must setup Redis and often separate instances to process the job queues. With Node and websockets in Ruby/Python, you typically must move any CPU task to jobs/task queues.
Sure, you may want to move your Java/Erlang jobs to Redis/SQS/etc for different reasons, but it is not your starting point. And given you were explicitly talking about CRUD apps, being able to have everything you need with only your app + database, without a need for Redis, workers, or tasks queues, leads to operational simplicity.
But part of the idea behind a queue job is not to overwhelm the server with heavy jobs that aren't urgent. Even with a thread pool, in java, you may still overwhelm the server with heavy jobs like sending emails, if you do not clog the threads you may still spike the memory - do you really want your server and your background jobs to share the same memory and resources?. So the separate instance make sense to me - they are a feature in fact. I really want to know from someone from the java ecosystem more about how they handle heavy background jobs. I get that Elixir may have this part done in a nicer way, but still ,Sidekiq is a VERY polished tool.
probably a bit of loose wording there. It's probably not what either of us would call objectively rare, but it is comparatively rare compared to many other runtimes. There's quite amount of this that can be avoided with some vertical or horizontal scaling depending on what you are doing with these events.
For instance you linked the spring application events integration with redis, while that is definitely an option it's not one you'd immediately reach for. Instead you'd more likely set up and use application events and/or domain events (if using spring data) handled locally with thread pool. The benefit here is that of course way less set up and complexity, but you can also handle these events sync or async.
"This thread pool is not well abstracted typically, and the mutable object model leads you to writing classically tricky multithreaded code with data races and the like" : If each request gets its own thread I don't really see the problem (they will not override each other's shared objects). Yes you sometimes run into stale reads/writes which you can solve with db locks etc.
Elixir can run concurrent background tasks without additional tooling. Can the technologies you mentioned do this (ie without setting up and maintaining a background job server, etc)? Ditto on pushing websockets. With Elixir, you don't need additional servers and caching for holding the the socket state (redis + node, etc). Hope that makes sense.
Elixir is a very cool and powerful language for building web apps.
Having built my first product (https://retrochat.xyz) in Elixir, I can wholeheartedly recommend it to anyone who is not happy with their current backend language(Ruby, Python, Js, etc).
Docs, functional programming, OTP, ecosystem(Phoenix, Ecto, Mix) are really top notch. In my opinion, you will be able to reach much higher productivity (especially long term) in Elixir than in Ruby, which is known for it's high level of productivity. Sounds like a fairy tale, right? Well, that fairy tale can become a reality with Elixir.
The only thing that is a bit of pain is a good release support, but that has been changing for better.