Flow/Broadway + how nice Ecto is already had me thinking, "Elixir is super interesting for data engineering." Explorer and Livebook have me thinking, "Elixir has the best shot at unseating Python."
Julia was designed to unsettle a creeping set of python in the domain of Perl, C++, and Matlab in scientific computing and various quant analysis that’s pretty distant, IMO, from what industry data engineering, ML, and analytics work was doing.
Doubt it. Elixir is a leaky abstraction over Erlang, so OTP concepts are a huge barrier of entry for non-software engineers.
e.g. to this day starting a small library requires learning supervisors, genservers, etc. Unless you’re doing a one off .exs, you’re going to be really confused by what “mix new” does.
Saying this as a passionate Elixirist — there’s nothing wrong with the leaky abstraction part, it’s what makes it powerful — but dethroning Python isn’t happening anytime soon.
We often tend to compare having to learn OTP with “not having to learn OTP” but the reality is that we need to compare OTP with concurrency tools in other languages (e.g. threads, locks, mutexes).
Of course, one can say:
“I wouldn’t use threads in other languages” but that’s potentially a bigger problem, as you end up with software which can’t easily exploit concurrency (which is especially important in the data domain).
I still think what you said has some truth to it, but I think the frame of reference is a bit off. I would rather learn a bit of OTP to get concurrency and fault tolerance rather than learn nothing and get none of it. :)
PS: “mix new” does not generate supervision trees by default.
> compare OTP with concurrency tools in other languages (e.g. threads, locks, mutexes)
That’s a very fair point, but is the average data scientist going to be learning this or offloading that task to a software engineer?
> I would rather learn a bit of OTP to get concurrency and fault tolerance rather than learn nothing and get none of it. :)
Agreed —- I’ve been using Elixir for a bit and am only now learning about supervision trees and genservers. It’s such an exciting part of the language.
> PS: “mix new” does not generate supervision trees by default.
Yup and that was confusing for me as a beginner — I recall being confused over where the entrypoint of the app was. Coming from a language like Python.
Hi José, huge fan. Thanks for dropping by this thread!
Maybe I've just forgotten what learning Elixir was like, but I feel like OTP never gave me all that much trouble? I guess it might be a little weird, but conceptually it's not that different from how objects work and presumably Python data people do fine with that. I feel like the sticking point might be the syntax. Elixir has a very developer-friendly syntax, but the flexibility (all the ways to call functions, implicit returns, arity stuff, collection syntax, even macros) makes it way harder to just learn "incantations" and be able to read other people's code.
