The name change came about because we found too many projects named "Lumen" also were launched around the same time or shortly thereafter.
Originally I envisioned a path towards creating a new WASM framework to build web applications using Elixir. Since then LiveView has come out and has made this less of a concern for us. While WASM compilation will be possible and we hope to backfill the runtime needs for web development the real focus and interest in this project for me is...
WASI compilation. With Firefly we will be able build self-contained binaries that will have all (many) of the benefits building applications with the BEAM and distributed systems. If WASI can run on something you should be able to compile your Elixir applications with Firefly and target your desired chipset.
We recently reached a milestone that we are hoping to release a blog post about it in the near future.
Forget WASI, I'm interested in this project as a way to efficiently run Elixir on microcontrollers. It's a great model for embedded programming (not surprising given that was what it was designed).
> The primary motivator for Firefly's development was the ability to compile Elixir applications that could target WebAssembly, enabling use of Elixir as a language for frontend development. It is also possible to use Firefly to target other platforms as well, by producing self-contained executables on platforms such as x86.
I'd be very very interested to hear follow up, on how Firefly does actors. I feel like there's so many potential ways to target wasm, but the high concurrency spirit of Beam has such unique flavor. I'd love to read in & hear that spirit is well preserved.
> as that is a part of the core semantics for BEAM languages.
It's a part of the semantics of the runtime:
- an actor is all but guaranteed to not bring down the runtime
- an actor is all but guaranteed to never affect other actors
- runtime knows how put processes to sleep until the message they listen to arrives. This means all functions are re-entrant. Well, any process is put to sleep after a cretain number of reductions so that no process takes away time from other processes.
- runtime all but guarantees that process errors are a) isolated and b) propagated. That is when a process dies all other processes that monitor it are guaranteed to receive a notification. That's why supervision hierarchies in Erlang are possible
> - runtime knows how put processes to sleep until the message they listen to arrives. This means all functions are re-entrant. Well, any process is put to sleep after a cretain number of reductions so that no process takes away time from other processes.
This paragraph seems confused. Re-entrancy doesn't have a lot to do with sleeping. Do you perhaps mean to say something about preemption? BEAM/ERTS is not really preemptive, a process can only be suspended at specific places, but one of those places is function calls (aka reductions) and BEAM languages don't offer looping constructs other than recursion, so it's hard to go for very long without calling a function, so it's effectively/semanticaly preemptive, unless there's naughty things in NIFs you brought, or the native code provided by ERTS.
Processes being descheduled after a while doesn't mean they didn't take away time from other processes: if you have one cpu and one process running an infinite loop, adding a second one takes cpu time away from the first, but they both will get some time (if they're both set to the same priority)
This is implied. It would be absurd to have a different static and runtime semantics. in fact, the core goal for formal methods is a statically be able to reason about runtime dynamics.
Just like it would be absurd to build a compiler for C where "+" in fact is treated as "-", it would be absurd to build a compiler and runtime system for, eg., Elixir that is not able to execute GenServers.
You'd be surprised how many people miss that runtime drives this. I've seen many discussions where people claimed "you can implement all this is a library" :)
> it would be absurd to build a compiler and runtime system for, eg., Elixir that is not able to execute GenServers.
> You'd be surprised how many people miss that runtime drives this. I've seen many discussions where people claimed "you can implement all this is a library" :)
I undrstand, everything concurrency is definitely not easy to implement in "User land" and is something you want good primitives for – why I am also amaxed over this project as they must have embedded that functionality (the scheduler) in the executable (Which they also say they did).
> Well, Akka did it on top of JVM ...
> Akka is a toolkit for building highly concurrent, distributed, and resilient message-driven applications for Java and Scala.
Akka does not seem to claim that they build a new runtime for BEAM languages?
You can't have those things without runtime support. E.g. in Go/Rust a panic will kill the app. In Erlang an equivalent catastorphic failure in a process will kill the process, will notify the monitoring processes, the app will keep on running.
Also the BEAM bytecode compiler and runtime are incredibly slow and unoptimized. It doesn't matter much for process handling and IO dominant workloads, but you would not want to run it with normal tasks.
An AOT compiler with better optimizations will run circles around BEAM on benchmarks.
> Also the BEAM bytecode compiler and runtime are incredibly slow and unoptimized
Can you provide any citations for the BEAM runtime being unoptimised? In my experience it has been very carefully optimised over many years, generally prioritising latency over throughput.
I'm more of an observer, since I'm not actively using Elixir, or Erlang, right now. I read that BEAM now supports JIT compilation. Doesn't this solve the performance issues for the most part?
EDIT: Apparently not LLVM JIT but that's beside the point.
BEAM does have a JIT on some platforms (iirc, amd64 and aarch64), but it's not an optimizing JIT like you might be familiar with from Java's Hotspot and similar systems.
In BEAM Asm, the design is for the whole VM to either be interpretted (status quo) or native (JIT). In JIT mode, all the loaded modules are translated to native code as they're loaded; this needs to be fast or startup times are delayed, IIRC, there is an optimization path, but it's simple. There's no reoptimization of hot code paths later either; just the one time process.
The main benefit of this process is to remove a specific part of interpretation overhead, the instruction unpacking and dispatch overhead is eliminated. This can be significant for some applications and not for others, but it's really the main target, any other optimizations that happen are a bonus.
If I were to try to do a straight shot at getting elixir on the frontend I'd just send bytecode up to the frontend and interpret it, either in js or wasm. Erlang bytecode is fairly straightforward, and especially if you stay single threaded you can make a reasonable approximation of the compute model[0]. The hardest parts I imagine would be coming up with an abstraction that lets you manipulate the DOM and test it while staying in the elixir test environment, which is insanely good. I don't see how firefly makes that particular challenge any easier.
I feel like the dockyard folks just got rust crazy and decided to do this -- for reasons[1], rust is honestly not the best language to do a BEAM implementation in, which is I think why they pivoted to a compiler
[0] except for the fact that a lot of libraries unnecessarily spawn processes.
You can build an app that runs entirely in the browser eliminating the need to have a server and reducing latency. Of course it doesn't make sense for all use cases but for some apps it might make a lot of sense. For example that is how Liveview Desktop works.
I'm a big fan of LiveView, but when you get to a very high degree of interactivity or anything that needs to work well offline or with bad signal quality then LiveView is not the right choice.
> The primary motivator for Firefly's development was the ability to compile Elixir applications that could target WebAssembly, enabling use of Elixir as a language for frontend development
A lot has changed since then.
The name change came about because we found too many projects named "Lumen" also were launched around the same time or shortly thereafter.
Originally I envisioned a path towards creating a new WASM framework to build web applications using Elixir. Since then LiveView has come out and has made this less of a concern for us. While WASM compilation will be possible and we hope to backfill the runtime needs for web development the real focus and interest in this project for me is...
WASI compilation. With Firefly we will be able build self-contained binaries that will have all (many) of the benefits building applications with the BEAM and distributed systems. If WASI can run on something you should be able to compile your Elixir applications with Firefly and target your desired chipset.
We recently reached a milestone that we are hoping to release a blog post about it in the near future.