From a "marketing strategy" point of view, I wonder what the thinking is in investing in this stuff so heavily when Python seems to be kind of the go-to? Will they be able to create a "good enough" environment to do that kind of work with Elixir? Is it just someone or a company scratching their own itch?
This is a genuine question - I don't know much about "AI stuff", but do know something about the economics of programming languages and I'm "intellectually curious" about what is driving this and what the goals are, rather than critical of Elixir. I love working with BEAM and miss it.
One factor may be that a few years back the language creator (José Valim, also the author of this article) announced that the language is basically "completed", and that they would shift focus to other things like developer tooling and other projects outside of the language itself.
José is quite prolific, so I think it's natural that he moves on to things like this. It's hard to know what reception will be like until you build it.
From a strictly "marketing" point of view, if you want to grow the language and ecosystem, it seems the successful move is to stake out a place where you're likely to win.
I think often this happens more or less by accident rather than conscious design - but think of something like PHP which made it really easy to whip up quick web pages, or Rails which drive Ruby adoption for a better, more structured, but still dynamic and quick web programming experience.
And I suppose part of those happy accidents are people just hacking on something they think is cool, so I wouldn't stress too much about the "marketing aspect". I'm just curious what drove it.
I have always considered helping the community grow into a diverse ecosystem to be my main responsibility (the Python community being a great example here).
This particular effort started because some people got together and realized that we could do it! Do it in a way that felt part of Elixir and not just a bunch of bindings to C libraries.
We honestly never had the expectation that we had to beat Python (otherwise we would simply not have started). Early on, we were not even sure if we could be better at one single thing. However, 3 years later, we do have features that would be quite hard or impossible to implement in Python. For example:
* Nx Serving - https://hexdocs.pm/nx/Nx.Serving.html - allows you to serve machine learning models, across nodes and GPUs, with concurrency, batching, and partitioning, and it has zero dependencies
* Livebook - https://livebook.dev - brings truly reproducible workflows (hard to achieve in Python due to mutability), smart cells, and other fresh ideas
* A more cohesive ecosystem - Nx, Scholar, Explorer, etc all play together, zero-copy and all, because they are the only players in town
Of course, there are also things that Python can do, that we cannot. The most obvious ones being:
* In Python, integration with C code is easier, and that matters a lot in this space. Python also allows C to call Python, and that's just not possible in the Erlang VM
* Huge ecosystem, everything happens in Python first
At the end of the day, what drives me is that the Erlang VM offers a unique set of features, and combining them with different problems have historically lead to interesting and elegant solutions. Which drives more people to join, experiment, run in production, and create new things.
Thanks for the great answer! And thanks for your work with Elixir. As a long-time Erlang user, I was impressed at how it wasn't just a slightly nicer syntax, but included genuine innovations and improvements.
My guess is that Jose and the core team are both personally interested in the big wave of ML stuff we've been experiencing recently and also want to demonstrate that Elixir is a viable platform for doing this work to teams which have adopted Elixir and are interested in ML but don't want to add a bunch of Python into their codebase.
I would also guess that Elixir led to some crazy things they didn’t even imagine possible with web, like phoenixs live view [1]. Even if they don’t have explicit ideas of how it will impact ML, it’ll be really interesting to try.
Correct. When I did the presentation, I did mention we attempted to type Elixir and failed. Therefore, the idea of saying the language was "complete" was precisely to signal the community that most of the work would happen in the ecosystem and not by directly changing the language. Especially because:
* The language was designed to extensible - see how Numerical Elixir can compile Elixir to the GPU and it didn't require changes to the language (we improved the language based on some needs but it was never a blocker)
* A lot of the hard work that makes Elixir better every year is actually happening on the Erlang VM, which we mostly get to leverage by doing nothing (for example, Elixir "got" a JIT since I said we were complete and we had to change nothing)
I'd say the definition of "completed" still holds, because we haven't announced changes to the language yet, but if the work on the type system succeeds, then we will introduce a new typing annotation, which is definitely big enough to void the label after 7-8 years.
Strictly optional and without changing the language API, fwiw. So about as smooth/painless an experience as possible.
Its addition into the next minor verision (1.17) will bring warnings that address some of the most common footguns in the language, like comparing structs.
I actually think the BEAM is an ideal environment for machine learning, sharding things across machines. The only thing I’m not sure of is if PyTorch etc. are more optimised than XLA the backend Axon uses… would be good to see some performance comparisons of a big LLM running on both. For everything else I’d suggest Elixir was a better experience.
It's very serious. The BEAM had a problem in that it lacked solid number-crunching libraries, so some folks solved that, and when they did distributed ML-capabilities kind of just fell out as a neat bonus so some folks did that too.
So now it's integrated into the basic Livebook, just boot it and go to that example and you have a transcriber or whatever as boilerplate to play around with. Want something else from Huggingface? Just switch out a couple of strings referencing that model and all the rest is sorted for you, except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.
Python is awful for sharding, concurrency, distribution, that kind of thing. With the BEAM you can trivially cluster with an instance on a dedicated GPU-outfitted machine that runs the LLM models or what have you and there you have named processes that control process pools for running queries and they'll be immediately available to any BEAM that clusters with it. Fine, you'll do some VPN or something that requires a bit of experience with networking, but compared to building a robust, distributed system in Python it's easy mode.
I don't know what the goals are, but I perceive the Nx/Bumblebee/BEAM platform as obviously better than Python for building production systems. There might be advantages to Python when creating and training models, I'm not sure, but if you already have the models and need to serve more than one, and want the latency to be low so the characteristically slow response feels a little faster, and don't already have a big Kubernetes system for running many Python applications in a distributed manner, then this is for you and it'll be better than good enough until you've created a rather large success.
> except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.
We use Elixir for our primary application, with a fair amount of Python code to manage our ML pipelines. But we also need real-time inference and it's really convenient/performant to be able to just do that in-app. So I, for one, am very grateful for the work that's been done provide the level of tooling in Elixir. It has worked quite well for us.
I can't say I know much about AI stuff or BEAM. But my best guess is that elixir native ML should integrate well with OTP's distributed computing capabilities. As an outsider to the elixir ecosystem, I've seen glimpses of elixir ML here and there but no mention of attempting to bridge the python ML ecosystem into elixir.
This is a genuine question - I don't know much about "AI stuff", but do know something about the economics of programming languages and I'm "intellectually curious" about what is driving this and what the goals are, rather than critical of Elixir. I love working with BEAM and miss it.