Two seconds to process a tiny text file enters well into the realm of "completely unusable" in my eyes.
It wouldn't be so bad if the Julia developers acknowledged that this is a valid concern (that they are not dealing with it right now for whatever reasons) and that the ecosystem will not be considered complete until this fundamental problem is solved. But this is infuriatingly not the case. Instead, they tell you that you are "holding it wrong" and that this is not really a problem, that your usage is "niche", that the interpreter is alright as it is, and that the time to first plot is unlikely to ever go below a hundred milliseconds. I find it really depressing for the language itself is incredibly beautiful. My only hope is that an independent, fast and unix-friendly implementation of the language arises, thus freeing the reference implementation of the efficiency burden and allowing it to be simpler. Something like the lua/luajit split.
I promise ecosystem will not be considered complete until this fundamental problem is solved. That said, you can have time-to-first plot below a hundred milliseconds right now if you put Plots into your system image - that's always been an option. System images have workflow issues which is why they're not used more.
I wonder where you got the impression that latency and precompilation performance are not valid concerns. This has been the _main_ focus area for the devs for a long time. It's pretty much all anyone has been talking about for over a year, and serious improvements have been made.
> I wonder where you got the impression that latency and precompilation performance are not valid concerns.
Now that you ask it, I realise it's been mostly through a few HN interactions! Every time I raised the issue in Julia posts over the last few years, I have been consistently ridiculed by purported Julia defenders. For example, in this very thread you can find a case of that.
It think that the discussions on HN tend to be quite combative, and members of the Julia community (including myself) can get a bit worked up over negative claims that we find overly sweeping. That could lead to some dismissiveness.
I do get frustrated by characterizations of the interests of the Julia community that are very broad, and are also in evidence here. E.g. somewhere upthread one person claimed Julians are only interested in Jupyter Notebook workflows, which is a totally alien statement to me. Jupyter doesn't seem to have a very large mindshare as far as I can tell, I've never even tried it myself.
Static compilation and analysis comes up regularly on Discourse, but among the scientist part of the community, there is naturally less emphasis on this. Please don't extrapolate too far based on some isolated interactions.
> the time to first plot is unlikely to ever go below a hundred milliseconds
How is that controversial or disappointing!? Why would anyone bother optimizing this? Nor is matlab/octave/python any faster.
`python3 -c "import matplotlib.pyplot as plt; plt.plot([1,2,3]);"` takes 600ms on my (powerful) workstation and that does not even include creating the plot window.
To be clear, I do believe there is much more work to be done to decrease latency in Julia, but your targets are ridiculous. And as a regular on their bugtracker and forum, the devs definitely acknowledge these issues and have many times said it is one of their main priorities.
By the way, if you want streaming live updated plots, this latency to first frame is not a problem. It is already straightforward to make such fast live plots in Julia (although it does not fit my personal taste for how to do it).
Come on, it is silly to compare a full general purpose language against a special-purpose tool. Yes, grep is also better than julia at searching for a string in a file.
Julia is a terrible replacement for gnuplot and gnuplot is a terrible replacement for julia.
Why is it such a ridiculous comparison? Gnuplot is still interpreting a language, loading plotting code, etc. If Julia folks wanted to, they could bundle pre-compiled plotting code that loads as fast as memory moves bytes. They don’t want to, of course, likely because it’s inelegant, but they could, and a general-purpose language doesn’t stop them from doing that.
You can already bundle pre-compiled plotting code in your Julia sys-image if you want. But Julia is not a plotting tool so it would be ridiculous to optimize it just for plotting. I want ODE solvers to have less latency, should I start expecting gnuplot to have built-in ODE solvers or the official installer of Julia to have the ODE libraries pre-compiled?
Maybe this example would make it clearer: why does your argument not apply to Python? Should we expect python libraries to come pre-cached so that the first time I load `sympy` I do not need to wait for tens of seconds to have .pyc files created. Or about matlab?
Again, I am all on board with the idea that julia needs lower latency and if you look at what their devs say, they also agree with that. But expecting Julia to be super low-latency (lower-latency than python/c/matlab) for your pet task is silly.
I gave a proof-of-concept argument as to why something doesn’t need to take as long straight out of the box with no customization. Python is doing it sub-1s. You can also include a non-optimizing interpreter. My point is that being a general purpose language doesn’t inherently limit you in any way; instead it’s one’s choice of implementation strategy.
Another strategy: when a user installs Julia, they select “fast-loading” libraries. You’d be surprise how small changes in UI/UX make huge perceived differences in quality and performance. I bet “Julia can already do this” too, but nobody does it because it’s not idiomatic and it’s not recommended up front.
At the end of the day, people don’t complain about Python or MATLAB as much because they feel nicer. If it feels nicer because of some other reason than absolute time, then they’re doing something about UX that Julia is not, because everybody really does feel Julia is extremely sluggish to use.
One thing that would help a lot is if julialang.org offered downloads with a bunch of common packages baked into a sysimage (modeled roughly of of anaconda). My rough estimate of packages to include would be
Plots, BandedMatrices, DifferentialEquations, CSV, DataFrames, Flux, PyCall, (and probably some others). Having a distribution with a lot of the heavy hitter built in would make workflows using those packages much snappier.
>Another strategy: when a user installs Julia, they select “fast-loading” libraries. You’d be surprise how small changes in UI/UX make huge perceived differences in quality and performance. I bet “Julia can already do this” too, but nobody does it because it’s not idiomatic and it’s not recommended up front.
Don't most people already use a bit of PackageCompiler.jl in their workflows? Maybe it just need to be mentioned more in introduction videos, but it's literally one line of code to do this and I don't see how that isn't the solution already. Most users I know have been doing it for a few years now.
For reference, with Plots and DifferentialEquations the command is:
using PackageCompiler; create_sysimage([:Plots,:DifferentialEquations])
It's simple enough to add to every tutorial. I think the issue is that intro videos just need to be updated.
I'm new to Julia. I had heard of PackageCompiler, but assumed that was only for creating redistributable binaries for others. I also assumed `]precompile` was the best I could do to speed up libraries.
My suggestions: 1- please forget about the intro videos. Those are impossible to keep up-to-date, and are not a good reference resource. Sure, a future video should explain PackageCompiler, but that is not sufficient. 2- please update docs.julialang.org with all such guidelines. This is the likely the first place people look, so if there are important guidelines that everyone could benefit from, this is where they belong.
Sure! That's why I set a somewhat reasonable target at being just 5x or 10x times slower than a specific-purpose plotting tool. But even that is considered to be chimerical! (or, in your words, "ridiculous")
Now you are just putting words in my mouth. I would completely agree that 10x latency to first plot is a reasonable target (i.e. first plot in about a second, like you get in python and much faster than what you get in matlab/mathematica). And plenty of devs closer to core of Julia and the plotting libraries in it would agree.
And to be clear, I do expect my second plot to be ready in tens of milliseconds.
Solved via flags to disable JIT and brought it down to a couple of secs. Native binary would be much nicer.