I'm currently writing code with it. The language is really (really) nice to work with.
- But the REPL lacks the ability to redefine structs on the go (which I can understand as it'd be tough to do, or simply not possible). But that, combined with the slow start up time makes life a bit harder than it should. Fortunately, one doesn't redefine its structs every day.
- There are also lots of libraries but the quality of the documentation is often sub par. For a newcomer like me, working examples would be great. For example, if you use the plots library, you'll have hard time finding a list of all possible plots (the documentation talks about lots of things, but strangely, not a list of possible charts). I've also looked at doing linear regression and GLM and, again, you've code libraries but the examples are rare...
- the startup time are still quite slow but that's ok because somehow you adapt your workflow around.
- being able to use greek symbols as identifiers is super cool but your editor as to manage that, else you'll have to memorize shortcuts...
But still, I keep on using it, it's much faster for my use case (data processing). I mean, faster than r or Python (for which I could write fast code but that'd mean I'd have to change the way it is written)
> There are also lots of libraries but the quality of the documentation is often sub par.
One thing I think R does not get enough credit for is really strong enforcement of documentation. If you want to get a package in CRAN it is going to be more work to not document your library than to just document it correctly. As a result nearly every R package has very solid documentation, including a well formatted pdf manual, a series of "Vignettes" that show how to use the library, and excellent in-interpreter documentation.
I don't know what libraries you're refering to. But most of the time, they barely document what they do. By barely I mean:
- parameters documentation can be understood only if you actually know the theory behind what the library documents. One may say that it's a good thing in the sense it prevents one to shoot himself in the foot but for discoverability, that's painful. I've done basic stuff such as GLM, LDA, PCA,... For example, there are several ways to do PCA's but which one do you choose ? Not everybody know the theory behind each PCA's formulation...
- examples are usually very limited and don't show what the library can do. For example, plots (the one from the base library) are really not well documented. Examples are really scarce and don't even show the graphics themselves.
- The package federation (not exactly documentation) is really bad : there are lots of library that overlap, that redefines symbols here and there without telling, etc.
So R documentation is so so... It feels like it suffers from lots of math explanation I've seen : a strong will to write the minimum possible which makes everything hard to get into.
I like R but documentation is not a strong point. Ecosystem is.
" I mean, faster than r or Python (for which I could write fast code but that'd mean I'd have to change the way it is written)"
You might benefit from numba. I've used it to speed my Python up enormously and completely painlessly just by adding decorators to critical functions. It's why I'm not considering moving to Julia.
There are so many ways to write R that can actually be very fast. R is a very different language as a whole and it has addressed a lot of problems these last 10 years. I might be biased but I really do like the functional side of R and how logical the libraries from Hadley Wickham have been designed.
Python still doesn't feel like a natural fix for data science work. I am guessing it is more bias opinion but why based on 0 for this domain????
> There are so many ways to write R that can actually be very fast.
But it means that you have to work with arrays. For me it often means breaking the flow of my (code) explanation to migrate to other data structures. Sure it is then fast, but it gets less readable and harder to update.
Now, I'm a programmer at heart, so I think in the "functional" paradigm, not the array/signal one. There's some "impedance" I guess :-)
I actually prefer the 0 indexing of python. Systems code (c/c++ etc.) already uses 0 based indexing. So it is nice that when you do data science the convention stays the same.
Does it actually matter at this point? I write off by one errors all the time in JavaScript and I usually spot them immediately. I’ve written Julia and R in the past and I found it really easy to switch between contexts, and if I forget, the errors are such that it takes only a few seconds to spot and fix them.
I think Numba has some pain points. I have also encountered issues with multi-dimensional arrays.
The beauty of Julia is that relatively naive Ruby-like code is already quite quick. And if you implement inner loops in an imperative way, with an eye towards not generating excessive allocations, it can approach C++ speed while still being nice high-level code that is close to mathematics or business logic.
Besides, the other strong point of Julia is composability. The ecosystem is made up by lots of small libraries that can interact in ways the original designers did not expect or plan for. In contrast, Python has exceptional libraries, but they tend to be big monoliths.
The problem with Julia right now is that some libraries are not sufficiently mature. For example, there's no mature native replacement for XLA or PyTorch. I know about Flux, but it's nowhere close if you wanna create, say, a large transformer. Or say you are working with GLMs. GLM.jl is nowhere close to R.
Some other Julia libraries represent the state of the art, though. I just can't wait to get all foundations complete! It's a really promising space for probabilistic and differentiable programming.
It just happened wasn't using Numpy except in the most basic possible ways. I had hand-written algorithms in python. That's where it shines. So, yes YMMV! It worked really well for me.
For simple things only.
For example, the jit class in Numba is still experimental, and quite limited.
So if you need to write non-trivial class, even if you jit each method,
dropping to Python can be a problem (say with lots of instances.)
I have tried to write an application targeting HPC
and in itself is not very complicated (probably ~2000 lines or in that order.)
But I did things like using the Python language as the metaprogramming language for Numba (basically higher-order function where you jit inside.)
All in all my experience of Numba tells me that if I am designing the same package
now I'd write it in Julia where jit is "first class" and you don't need to
constantly thing about the boundary between Numba and Python.
IMO, numba loses pretty much all of the advantages of python. The code is fast, but you lose the ability to organize your data the way you would in regular python, and most of the python ecosystem can't be called from within numba functions. If I wanted that developer experience, I would just use C.
I don't think that's true. You lose some functionality, but since you can call Numba on select critical methods, it's not so bad. You can often sequester your performance-critical logic into some Numba-decorated methods, and your business elements that call out to the rest of your ecosystem are not decorated with Numba.
Or to put it another way-- if I'm using vectorized code in Numpy I can't deal with the external ecosystem from within my vectorized code either.
> The code is fast, but you lose the ability to organize your data the way you would in regular python,
Can you clarify what you mean by this? I lose the ability to organize my data the way I'd like if I'm vectorizing my code too.
I wasn't comparing numba to numpy. I was comparing to python code where you don't care about performance. The main reason I don't find numba appealing is that Julia gives you numba like performance while allowing you to use structs (think classes) to organize stuff.
This doesn't really make sense. If you don't care about performance then you have no use for Numba.
Also, the comment you replied to was explicitly comparing Numba to vectorized Python, so you should not abandon that comparison in your reply without saying so.
I think the point is that one wants to write code that is similar to regular non-performance sensitive python code, with classes and everything, and still have it be fast.
> Also, the comment you replied to was explicitly comparing Numba to vectorized Python, so you should not abandon that comparison in your reply without saying so.
The comment you replied to explicitly says "regular python".
Not true at all for me, because in my situation there are some functions which have tight loops that take 99% or more of the execution time. Put the numba decorator in front of them and the code is 100+X faster. Clearly it's not that simple for everyone. But for some people, it can be. So it doesn't make sense to call it a panacea, and it also doesn't make sense to say it doesn't work. It works for some people, not for others.
The C API for Julia also has almost no documentation. There is a getting started guide, which is great, but if you want to do anything more advanced (e.g. creating structs like in your example), you'll end up reading the source code to try to puzzle through which functions to use in julia.h. There's also an apparent limitation that whichever thread initializes Julia is the only one that can later eval code, which was surprising. The language itself is very cool, but it has a long way to go to be easy to embed like Python is.
The good news is that you don't need the C api nearly as much because you don't have to call out to C whenever you want performance. Also, the extent to which the python C api is documented has actually been a major problem for them since it has effectively frozen a ton of python's implementation in majorly detrimental ways (eg the GIL)
Calling out to c "because you want performance" is only only one dimension of the issue, and assumes that your main application code is written in python or julia. In many cases (e.g., robotics), application code is written in c++ or c, and python bindings serve as simulation harnesses and visualization tools. Pybind11 is absolutely brilliant for this. The last time I looked, similar tooling for Julia was substantially less mature and definitely didn't look like something I'd want integrated into a production workflow.
> But the REPL lacks the ability to redefine structs on the go (which I can understand as it'd be tough to do, or simply not possible). But that, combined with the slow start up time makes life a bit harder than it should. Fortunately, one doesn't redefine its structs every day.
It is possible to redefine structs in Pluto.jl which is also a productivity booster overall due to its reactivity.
> For example, if you use the plots library, you'll have hard time finding a list of all possible plots (the documentation talks about lots of things, but strangely, not a list of possible charts).
All valid complaints. Regarding the first one, redefining structs, you can wrap them inside modules and reload the module. It's not as ergonomic since you will need to qualify the structs with the module name.
Can't you just export the struct?
Edit: Just tried it and ran into those difficulties. Julia structs appear to be treated as constants, and Revise.jl didn't help (for me).
I don't know why you cannot "overwrite" existing structs but to be fair whatever system you use to program in a stateful manner in the REPL will have some problems.
Just hit this one: after rewriting a function I don't see any change in behavior. Reason: I overwrote the function but the more specialized one was being called.
But there are plenty more. After some time it's just better to nuke the REPL and start clean. That said, I love programming in the REPL.
> Just hit this one: after rewriting a function I don't see any change in behavior. Reason: I overwrote the function but the more specialized one was being called.
Yeah, I've had this and the struct redefinition problem since the very early days of Julia, that's why I never fully bought into the Revise.jl based development model (it has its good parts, but these are big limitations that should be mentioned more often when recommending it). That's also why I resisted the removal of the `workspace()`-clearing function (like MATLAB's `clear`), since that would be an alternate option for quick and dirty exploration in a lot of cases; though these days the latency problems of exiting to shell and coming back are much less, so it's not as much of an issue.
Revise.jl properly handles the deletion of methods. If the person you were replying to were tracking their changes with Revise, it wouldn't have happened.
Yeah, my comment was confused in a way - I didn't notice they weren't using Revise.jl, but I was also talking about earlier versions of Revise.jl. Around the time that `workspace` function was retired (v0.7/1.0 times), IIRC, Revise.jl did have problems deleting methods, and that made it a big pain point in a language where method dispatch is such a central pattern. So it fell short as a replacement for workspace-deletion function, despite being suggested as one, and that frustrated me. I'm probably holding on to that negative impression for far too long though, by this point; I'll make a more wholehearted attempt at using a Revise-based workflow and see how it feels today.
Examples and tutorials should go first. Some parts of the ecosystem, for some reason, put the manual and descriptions first. Give people code, then give people a manual for if they want to dig further. Thankfully, this is rather easy to fix.
I cannot remember exactly when I discovered Julia or where. But I remember I got intrigued early on and wrote a blogpost about 9 years ago on Tumblr, which is still there comparing Julia to Ruby of all things: https://assoc.tumblr.com/post/70484963303/getting-comfortabl...
The normal thing today is comparing Julia to R, Matlab and Python. But my intro to Julia was actually trying to convert code examples in the O'Reilly book: "Exploring Everyday Things with R and Ruby: Learning About Everyday Things" to Julia.
Unlike your typical Julia user I was always into programming because I wanted to make computer games. I liked playing with Game engine architectures and those exposed me to the problem of handling collision between objects of different types. Multiple-dispatch solves that problem very elegantly. Traditional object-oriented languages are pretty bad at it.
From the perspective of writing clean and expressive code, this really won me over to Julia early on before I had done much serious work with it.
It became my goto language for writing Unix tools, replacing Go, Ruby and Python.
Hehehe oh yeah, I almost forget about that because I stopped years ago bothering to actually make the tools into separate files that I start from the shell. It is more like Julia is my shell. I'll just have packages with common tools I use and launch them by calling functions from the REPL.
I'll just have packages for doing doing image file conversions in batch, modifying source code, changing configuration files etc.
Actually come to think about it, I used to have shell scripts in a pipeline to build an application for security. We had to obfuscate the source code and stuff like that. When I rewrote to Julia it all ran much faster. Even if there was a startup time, the workload was heavy enough that the higher performance of Julia easily outperformed bash.
Aren't many bash programs written in C? So this is implying Julia is somehow faster than C? Obviously that can't quite be true, but I can definitely imagine that the algorithms implemented in Julia could be fast faster than other algorithms - since the community has such a heavy influence of very hardcore mathematicians that have a string stress towards speed. Probably most of the algorithms in Julia are state of the art and push the boundaries on time complexity.
I was talking about bash scripts. But outperforming C with Julia is perfectly possible. Julia JIT compilation means you can remove overhead of a lot of function calls which C cannot do. A simple example would be sort taking a function pointer doing object comparison.
High level functional style code with things like map and filter can frequently be JIT compiled to optimal machine code.
Fortran is considered faster for numerical code than C and well polished Fortran libraries like BLAS is already getting outperformed by Julia.
For typical systems programming with need to tight control of memory and real time system C will still have the edge. But for anything crunching lots of numbers like data analysis or machine learning Julia will likely outperform everybody else.
And the "how" behind Octavian.jl is basically LoopVectorization.jl [1], which helps make optimal use of your CPU's SIMD instructions.
Currently there can some nontrivial compilation latency with this approach, but since LV ultimately emits custom LLVM it's actually perfectly compatible with StaticCompiler.jl [2] following Mason's rewrite, so stay tuned on that front.
Thanks. But how LoopVectorization.jl is helping here, say comparing to C/Fortran
optimized w.r.t. to the CPU? Is there somewhere in their doc mentioning this?
The basic answer is that LLVM doesn't do as good a job with some types of vectorization because it is working on a lower level representation. There are several causes of this. One is that LoopVectorization has permission to replace elementary functions with hand-written vectorized equivalents, another is that it does a better job using gather/scatter instructions.
The list comprehension in your example is now type-stable without additional type-annotation so you can just write [x^2 for x in xs] and the compiler known that the results is a list of integers (for your example). There have been many improvements of this type in julia over the years (kudos to all who made this happen).
why ruby "of all things"? i've only dabbled a little in julia, but it definitely has a lot of the things i like about ruby, mixed in with a lot of the things i like about racket.
Not OP but I read the next sentence as explaining that sentiment:
> The normal thing today is comparing Julia to R, Matlab and Python.
I'm on the same page as you, though. I've never written R or any Matlab, but when Julia was still new, I remember seeing an example that it was a language that didn't use S-expressions, yet had as potent of macro and meta-programming abilities as if it did. Seemed like it'd be wonderful to work in.
Yeah it was stuff like that which attracted me to Julia I just did some meta programming to allow calling Objective-C from Julia using ObjC like syntax thanks to macros.
Most of the time multiple dispatch and a what really help abstracting your code and reuse.
I used to love how everything was an object with methods in Ruby, but now I have come to prefer the more functional Julia approach. Free functions compose so much better when doing functional style coding than methods on objects.
I started using Julia in 2017, and as a physics student who saw Python, Mathematica, and Matlab as a series of tools that I had to pull out sometimes to solve specific problems, it came as a great shock to me that I was suddenly interested in Julia for it's own sake.
It's not an overstatement to say that this language totally changed my relationship with computers and programming.
I can reflect on like past few years (like 1.5 year ago i was disappointed by julia lack of progres but a lot have changed).
Julia is getting usable even in "normal" applications not only academic stuff, as person who come back after 1.5/2 years to julia i feel like i can use it again in my job cause it is a lot more stable at have a lot of new neat futures + CUDA.jl is amazing.
I hope Julia team will still explore a bit more static type inference and full AOT compilation if language got full support for AOT it'll be a perfect deal for me :).
Yeah, it's definitely in the early stages still, but this time I think there's much more infrastructure, and more people around with the right knowledge to advise on StaticCompiler's development, that I'm currently feeling pretty good about it's future.
I recently made some time to come back to Julia and it's been wonderful. Unlike the early frustrating days when major version updates broke so many things, the core is solid and the ecosystem is wonderful. I am so happy learning and programming that I have to stop myself from spending too much time messing with it. Kudos to the Julia developers!
I've been working through the Statistical Rethinking[0] course with Julia recently and so far the language has been really intuitive, same for the libraries (Turing, Distributions). And it's nice that you don't have to vectorize your code and can use normal for-loops.
Also, the way dynamic dispatch + specialization work is beautiful.
Julia is a very pleasant language to work with. The type system, multiple dispatch, and package system all make it into top 10 of design.
The only change I would recommend is to have a way to distinguish between creating and setting a variable. Without this distinction it is very easy (especially for someone with dyslexia) to misspell a variable and accidentally create a new one instead of assigning a value. Something like this would be nice:
Perhaps this is something that could be fixed with a tooling, rather than language change? E.g. Syntax highlight a different a colour if it's the first use of a variable.
It may sound dumb, but such a "simple" change (simple conceptually, can't speak of the dev challenge involved) change would be an enormous quality of life increase to me. I've ended up looking into scoping rules and what not when bug-hunting because of this, but often just ended up going back to a language with stricter syntax around variable creation (goes for moving away from Python etc as well). The more the code, the bigger of an issue this becomes.
Would you be willing to provide an example where this becomes a problem? Probably unpopular opinion, but I don‘t think this should be a problem for structured applications and hints at a more fundamental design problem.
> Would you be willing to provide an example where this becomes a problem?
It's not a specific example, as much as every piece of code where mutability is involved. I don't have a LOC number in mind, but it could be for blocks/local scopes. IMO syntactic explicitness helps clarity and thus bug hunting/maintenance, and I feel tooling shouldn't be an alternative for dealing with, albeit subjective, syntactic shortcomings.
> this should be a problem for structured applications and hints at a more fundamental design problem
Frankly, I don't see how introducing an explicit hint that a variable is created can ever be a bad thing. Explicit lack of mutability is also something I'd like to have (e.g. 'let' vs 'var') - sometimes I want to make sure not to accidentally change a value. We will shoot ourselves in the foot at some point, so a syntax that allows for a check whether I can actually change a value further down can only be a good thing.
I absolutely do not trust myself. :)
When things go wrong for this reason (difficult to spot typo etc) I may waste time on investigating whether blocks are actually leaky or not etc - I have issues with Python syntax for the same reason. Going in the other direction, I like Rust's perhaps overly verbose "let mut" much better, since this has other consequences for explicitness (mutable references '&mut X' vs immutable references '&X'). No special tooling or IDE required to spot this, I can 'cat' the code in a terminal and it's still obvious.
I absolutely don't think Julia or Python should "go Rust", since their dynamic nature is a strength, especially so in academia where notebooks are popular. But a simple "I'm creating a variable" should IMO be a nice visual cue to any developer.
Thanks for your answer! I think it‘s a tradeoff between syntactic simplicity and explicitness. I enjoy languages like Scala or Rust that are more explicit, but I enjoy Julia as well (partly because of its simplicity).
It comes up all the time 100 line one-off scripts in python. You start off with 10 lines that you're iterating on. You move some snippet to a function, don't de-dupe your variable names from the global scope, and you get a bug where you thought "x" was scoped to the function, but wasn't actually defined in the function so you're using the global "x". Yes, this could be fixed by moving the global statements into a main() function, but this breaks your repl workflow and you have to resort to breakpoints in main().
This has been discussed before, and the tl;dr is that it's unlikely to happen. The discussion here[1] gives a sample of the implications and thought processes behind it, considering what it would mean to have := in different contexts, but the conclusion was:
> It’s just an interesting thought experiment — we can’t practically do this even in a 2.0 release. It would be a massively breaking change: we would need to disallow implicit declaration of locals by assignment, which breaks all Julia code everywhere. I also think that what we do now is good and less annoying for new users since they will be accustomed to just assigning to initialize and declare new locals if they are coming from Python, R, Matlab or Ruby (to mention just a few).
(What isn't mentioned explicitly is that this would also break a lot of macro code which accepts assignment-like syntax, which would have to deal with this new type of node in the AST and make decisions about how to handle it.)
The good news is that, as Stefan mentions in a thread linked from there, you can already do `local mylongvarname = 5` as an explicit variable creation syntax; and Julia has good enough tooling to parse it (and is getting even better ones as we speak) that you can disallow assignment to any variable that you haven't seen before with a `local` declaration, with a linter-like step. So the linter will error on a later `mylngvarname = 42`.
It's not the language itself adding it as a feature - so code inherited from elsewhere likely won't have it, and you might have to get buy-in from your team - but it's probably as close as you can get to what you want.
I started playing with Julia in 2013, after searching for a Matlab replacement that was (according to my notes at the time) "... open (or more open than Matlab), fast, parallelizable, with good scientific/math libraries." My intent was to write a reusable and extensible library to enable my day to day work (astrodynamics and estimation). Two years later I had implemented enough tools and gained enough proficiency with it that I switched entirely away from using Matlab at all (even for smaller data analysis tasks), and I haven't looked back.
It has made me objectively better at my job, and a much better programmer overall.
I first learned about Julia somewhere in 2013, I thought “damn, this looks really nice”, but never used it seriously until 2016 (v0.4). This language gives me joy, I never get tired of it.
Julia, because fast can be beautiful. I don't know how much it's going to grow but it doesn't look like stalling in the near future. The reason is simple: it fulfills a need no other language does.
I would guess there are some possibilities that could be a menace:
1. Someone manages to convince scientists to use a statically typed language.
2. Computing paradigm changes drastically. Everyone moves to quantum computing or similar.
3. In another 10 years someone learns from the lessons of Julia and improves on them to make a much better language.
I count myself as someone surprised at Julia's success.
As a replacement for Python, I've never understood the appeal, and it's probably not going to fill that niche. Still, as a replacement for Matlab/Mathematica, it's doing swimmingly.
"Replacement for X" is a vague phrase - will Julia take over every usecase of Python? Unlikely. Will projects with established Python codebases, that work well for them, suddenly switch to Julia? Nah.
Will new projects that need a flexible performant language, that would have otherwise been shoehorned into Python as the closest available option, choose Julia instead? Quite likely, and that happens a lot.
Python has its place, and is good at a lot of things. But there have always been a lot of projects that have straddled the line between numeric computing and general purpose computing, needing both, where Python was chosen for lack of choice, just because it was the best among the bad options. That's the pain point Julia addresses, and the people who have experience being stuck in that situation understand what its place is and why it has the success it has. (And being a well designed language that's a pleasure to use helps too.)
I don't really understand this comment. It was never intended to be a replacement for python in general use; do you mean as a numpy/scipy replacement?
In that case, I guess we'll see. Python will never be a particularly good language for implementing such things, but has become pretty ok for using them once someone has implemented (in another language, and made the python bindings).
Network effect is the real barrier, python itself is very easy to toss for this use .
I really, really wanted to like Julia. The syntax looked perfect to me (coming from scheme), and somehow everything felt 'right'. That's when I discovered that compiling to binary seemed to be frowned upon. I found some documentation on how to do this back when 1.0 came out, but it looked unnecessarily complicated and 'third-party'. I'm guessing Julia really is for sci-comp and data scientists, and not for producing redistributable bins. Does anybody know if Julia will ever become more 'general purpose', like say Racket or Python? Thank you.
I don't think it's frowned upon to compile, many people want this capability as well. If you had a program that could be proven to use no dynamic dispatch it would probably be feasible to compile it as a static binary. But as long as you have a tiny bit of dynamic behavior, you need the Julia runtime so currently a binary will be very large, with lots of theoretically unnecessary libraries bundled into it. There are already efforts like GPUCompiler[1] that do fixed-type compilation, there will be more in this space in the future.
Yeah, not frowned upon at all. StaticCompiler.jl [1] has made huge strides in the past few weeks actually thanks to Mason and Valentin (swapping in GPUCompiler as the backend); you can even compile to a tiny standalone binary without linking to the runtime if you're willing to use some tricks e.g. [2] to avoid GC allocations (stack allocations and manual heap allocations are both fine):
# This is all StaticCompiler-friendly
using StaticTools
function print_args(argc::Int, argv::Ptr{Ptr{UInt8}})
# c"..." lets you construct statically-sized, stack allocated `StaticString`s
# We also have m"..." and MallocString if you want the same thing but on the heap
printf(c"Argument count is %d:\n", argc)
for i=1:argc
# iᵗʰ input argument string
pᵢ = unsafe_load(argv, i) # Get pointer
strᵢ = MallocString(pᵢ) # Can wrap to get high-level interface
println(strᵢ)
# No need to `free` since we didn't allocate this memory
end
println(c"That was fun, see you next time!")
return 0
end
# Compile executable
using StaticCompiler # `] add https://github.com/tshort/StaticCompiler.jl` to get latest master
filepath = compile_executable(print_args, (Int64, Ptr{Ptr{UInt8}}), "./")
yielding:
shell> ./print_args 1 2 3 4 5.0 foo
Argument count is 7:
./print_args
1
2
3
4
5.0
foo
That was fun, see you next time!
shell> hyperfine './print_args hello there'
Benchmark 1: ./print_args hello there
Time (mean ± σ): 2.2 ms ± 0.5 ms [User: 0.8 ms, System: 0.0 ms]
Range (min … max): 1.5 ms … 5.5 ms 564 runs
Warning: Command took less than 5 ms to complete. Results might be inaccurate.
shell> ls -lh print_args
-rwxr-xr-x 1 user staff 8.5K Feb 10 02:36 print_args
GC allocations are allowed if you use instead the approach in this PR [3], but we haven't wrangled that approach to produce standalone binaries yet.
I'm hoping Julia gets its killer app that can launch it to the next level like Rails or Numpy. Julia has a lot of pleasantries, but not enough to pry me away from what I'm productive with. There's certain amount of switching cost that needs to be overcome.
Matrix operations are already built-ins in Julia. And it's such a pleasant to work with compared to the syntax of Numpy. People just don't want to get away from the familiar thing (i.e. Python).
And I don't think making a web framework would be a focus of Julia since it was built with science and computing mindset.
This might be cliché but use right tools for the job. I still use Ruby for web development and Julia for the science and machine learning stuffs (gradually moved from Python ecosystem).
one thing people don't see is how "free" things are in Julia... we don't need half a million line CMake and C++ then bind to Python to make something work, and we don't need monorepos that re-invent same interface (array, numpy, autodiff etc.) three times. It just works™, and small libraries constantly build on top of something with only ~100 lines and gives you something fast and flexible.
Pluto is that app for me coupled with a simple portable install.
Installing on windows (workday I'm consigned to flames of woe) is simply a matter of downloading portable install .zip file, a simple startup .bat script[0], 'add Pluto', 'import Pluto' and finally 'Pluto.run()'
I now have an Observable-like[1]notebook with all the Julialang goodness without running an installer
[0] my julia.bat looks like:
@echo off
@REM see https://docs.julialang.org/en/v1/manual/environment-variables/
@REM %%1 is the project directory
set JULIA_PROJECT=%1
set JULIA_DEPOT_PATH=%~dp0JULIA_DEPOT
set JULIA_BINDIR=%~dp0julia-1.7.0\bin
set PATH=%JULIA_BINDIR%;%PATH%
%JULIA_BINDIR%\julia
For me, that killer app was the JuMP package [1] for formulating optimization models. It has a nice syntax (via macros), connects to many solver backends and even allows for solver-generic callbacks. A list of features that is (mostly) unheard of in the Python world.
I didn't read every testimonial but I read a lot, and I have to say a lot of it resonated with me and reminded me of when I first came across Julia. Congrats on the 10 year journey, and I hope for more great progress in the future.
I get a sense that the julia community is more anxious about adoption than, e.g. python or R communities. Its probably natural given it is a relative newcomer in the broader "data science" / "scientific computing" thing and in the past years there was an explosion of interest / hype around some of its subsets (in particular anything that can be labelled machine learning or AI)
But participating in that "hype" is not necessarily what will entrench julia for the long term. Turning its unique characteristics (unique versus these other two open source contestants, not across the entire programming language landscape) into unmissable developer / user experiences seems to me a safer route. E.g what makes R impossible to ignore is the richness of its statistical toolkit. What makes python impossible to ignore is the productivity boost for typical tasks etc.
I thought Chris Rackauckas was particularly good. I think he ends with a good summary of one of the many reasons solving the dual language problem is important:
> There are so many places in math that are simply untouched because they sound like the domain of a compilers instead of "computational science", and I'm excited to see how the next 10 years bridges this gap in Julia.
Jose Storopoli (@storopoli): "I've made amazing friendships here, co-authored a free open access and open source Julia Data Science book with Rik Huijzer and Lazaro Alonso."
I just read this book. Every page was like, 'wow'.
A couple of questions I had afterwards:
* I saw the DateTime type, but it doesn't seem to have a timezone - how do you deal with timezones?
* How is it to work with async code?
* It's a garbage collected language, what are some techniques to avoid performance pitfalls due to that?
If there's other Julia programmers around, any answers would be much appreciated!
For timezones the official[1] answer is to use TimeZones.jl.
For async code Julia has "tasks" which are similar to goroutines in Go: they can be multiplexed on a number of OS threads for parallelism, and channels can be used to communicate with other tasks. The language provides some sugar to make some common things easier, for example to get the result of a long-running operation:
# Make channel and pass it to anonymous function
result = Channel() do chan
# Anonymous function body
sleep(5)
put!(chan, "Done")
end
# Do stuff
...
# Now get the result
take!(result)
When GC is a problem, the most common is to minimize allocations and disable GC at critical times. See [2] for a robotics example where some kind of realtime is required. The language provides tools to help diagnose allocations, and minimizing allocations is a popular sport in the community so it's easy to get help on that topic. Still, realtime is currently not where the language shines.
I really like julia. For certain problems matlab and python are unworkable. People complain about time to first plot, but that doesn't matter too much to me. The problem is that plotting in general is just not as robust or user friendly as Matlab or matplotlib in python. You can call matplotlib, But it feels like a second class citizen. Plotting just needs to be better with a clear preferred method. There are tons of options.
I agree, there are a variety of plotting library options in Julia but they aren't nearly as developed/robust as matlab (e.g. quiver plots). There are a number of plotting functions that have completely changed how they work across versions and generally lack descriptive documentation. There have been several occasions where I had to give up and export my data into matlab to generate a plot. I also think the image analysis libraries are underdeveloped.
All of these have some exceptions, but are for the most part true:
high energy physics
lots of computational bio
fluid dynamics
rendering
In general, python and matlab really struggle in problems where you want maximum performance, but the most efficient algorithms aren't vectorized. In some cases, this is solvable by writing python libraries in C/C++, but especially in scientific fields, the end users are often the same people writing the algorithms, so they don't gain much from a python library if they have to do all the hard work in C++ anyway. Julia gives them a way more productive dev experience while still having a good user experience.
indeed, the grants (in the millions $) given to rewrite C++ or Python just to handle array (because for loop sucks) and ends up making monorepo blobs is jaw dropping -- while it's almost free in Julia, with 1/100 of the line of code and more flexibility we can match and beat even C++ code...
Good for exploratory programming but trying to make a single binary for any re-distributable app is a nightmare. PackageCompiler will always beat you up. And frankly, why can't Julia have a simple, single CLI command to create a binary like any other PL.
"Any other PL"? Really? Well, that is news to me:)
Anyway, the answer to your question is very simple: Because it's difficult, and a lot of work. Not because they can't be bothered, as your question seems to imply.
Yes. There are good attempts towards static compilation, and we will have it one day, but it's something that requires high levels of expertise and effort given the dynamic nature and design of Julia - not just a matter of "doing it like any other language".
* TTFP - this is the big one. Every thing is a sluggish sometimes
* Sluggish IDE in VSCode; I am on Windows so start up is a bit of a pain i often have to wait 20+ s for it to become responsive
* Non-robust Basic Data Science coverage e.g. GLM fails on some datasets that R can handle. So it's the only code in my workflow that still has R in it
There is something elegant about having all functions at the top level (or module) scope, `foo(x, y, z)`, but I there is also something really nice about function names scoped to the thing/noun/subject: `x.foo(a, b)`. As far as I know Julia only does the former. Sometimes the latter seems a lot easier to deal with.
I think the design trade-off was to aid with multiple dispatch, one of the core paradigms in Julia. There, the implicit bind in `x.foo(y, z)` does not make `foo` useful for multiple dispatching, so their hand was forced. They may have gotten this from the Common Lisp Object System.
I think you mean you can do `foo(x, y, z)` instead of `M.foo(x, y, z)`.
You cannot do `x.foo(y, z)` instead of `foo(x, y, z)`, a feature I sorely miss in Julia. Nim is awesome in this regard. There are some functions / paradigms that are just better represented by `x.foo(y, z)`. Chaining functions in particular in typescript / javascript / rust is SOOO nice. Specifically, in Julia a lot of times I want to create a immutable struct with multiple fields, many of which have defaults. In Julia none of ways seem clean to accomplish this. In rust, something like this is very common:
let m = App::new("My Program")
.author("Me, me@mail.com")
.version("1.0.2")
.about("Explains in brief what the program does")
.arg(
Arg::new("in_file").index(1)
)
In Julia the same thing is:
m = App("My program")
m = author(m, "Me, me@mail.com")
m = version(m, "1.0.2")
m = about(m, "Explains in brief what the program does")
m = arg(m, index(Arg("in_file"), 1))
I think with improvements to the pipe operator this can be better but right now I find this painful to read / write.
> immutable struct with multiple fields, many of which have defaults
With `Base.@kwdef` (or the enhanced `@with_kw` from Parameters.jl) on the struct definition, that example can be:
m = App(name = "My program",
author = "Me, me@mail.com",
version = "1.0.2",
about = "Explains in brief what the program does",
arg = index(Arg("in_file"), 1)
which seems pretty nice to me.
Also, there's Chain.jl for a better pipe operator, though I agree improvements to the Base pipe would be great.
To the original point, the problem with x.foo(y, z) is that it would be highly misleading in that it seems to suggest that x is somehow special to the dispatch of the call, which isn't true. While superficially it can feel more familiar, it would instead become a trap for people giving them a wrong mental model of the call's semantics.
Unless you have some very weird use case (and probably even then), that sounds like a bad application of the "keyword arguments bad for performance" mental model.
The reason to use immutable structs is better performance when you use them, not when you construct them - how often are you constructing new struct objects that the constructor performance is a significant issue? And if you are constructing many many thousands of objects to the level it becomes a performance concern, any difference from keyword arguments will be easily dwarfed by the cost of actually constructing the struct in memory.
Also, these constructors ar simply passing on these named arguments to the non-keyword constructors. Anything complex you're doing during construction will be happening in the non-keyword constructors, and those do get specialized to have good performance; these keyword constructors are simply providing a small, simple interface to them, in a way that shouldn't affect performance in any measurable way.
Since keyword arguments don't participate in multiple dispatch, they can't be used for compile-time specialization of methods based on their type. That can be a performance issue in specific scenarios, but I think they've taken that as a general rule and misapplied it here (as I mentioned in my sibling comment). For this scenario, that aspect of keyword arguments is most likely irrelevant.
Make the package management more Cargo-like and I'd probably adopt it wholesale; whenever I've tried Julia in the past the user experience of just getting started has been the off-putting part.
> It takes a bit time to get into because it merges the concept of packages and environments.
Could you explain what you mean here? The point of environments is to manage packages and dependencies, so by definition they're going to be closely coupled. It seems something about it seemed unintuitive to you though, so I'd like to understand what it was.
Julia's package management was inspired by Cargo, afaik. What did you find off-putting about it, what would make it more Cargo-like? I've heard from people who are not used to the REPL wanting to do things from the shell directly, if that's the problem, there's the jlpkg tool [1].
The REPL was indeed part of my issue, though having tried again in the last couple of days following this post, my impression is much more positive than on my last check (around when v1.5 was new, I think).
Also, Pluto is a marvel - seems to fix all of my complaints about Jupyter and is trivially easy to use.
- But the REPL lacks the ability to redefine structs on the go (which I can understand as it'd be tough to do, or simply not possible). But that, combined with the slow start up time makes life a bit harder than it should. Fortunately, one doesn't redefine its structs every day.
- There are also lots of libraries but the quality of the documentation is often sub par. For a newcomer like me, working examples would be great. For example, if you use the plots library, you'll have hard time finding a list of all possible plots (the documentation talks about lots of things, but strangely, not a list of possible charts). I've also looked at doing linear regression and GLM and, again, you've code libraries but the examples are rare...
- the startup time are still quite slow but that's ok because somehow you adapt your workflow around.
- being able to use greek symbols as identifiers is super cool but your editor as to manage that, else you'll have to memorize shortcuts...
But still, I keep on using it, it's much faster for my use case (data processing). I mean, faster than r or Python (for which I could write fast code but that'd mean I'd have to change the way it is written)