Hacker News new | past | comments | ask | show | jobs | submit login

How?



What do you mean how? Something in base changes, the author of a package didn't put strict enough compatibility on their package and now you can't use the package. It's especially great when this sort of thing happens after spending ten minutes waiting for precompilation an hour for your time to first gradient and three days for a run to complete and you trusted the tool to let you write the results to a CSV using the CSV package(one of the most widely used packages). Then you file a bug report and get gas lit, so you just patch your local Julia version and compile it yourself so you don't wait 4 months for an upstream fix. Good times.


Counter anecdote, I had to go back to a quite complex project that a was developed in 1.5, 3 years ago, hundreds of dependencies.

Upgrading to 1.9 RC2 required changing a single dep (Light raps to the drop-in Graphs) and a single line (a library changed the return from a String to a StringView)


You're incredibly lucky. Even minor version bumps have left me up shit creek in production with deadlines.


Julia core is backwards compatible all the way to 1.0, libs not so much.

Now, something as fundamental as numpy dropped support for Python<3.6 in a minor release: https://numpy.org/devdocs/release/1.19.0-notes.html

And I've been bitten by relevant python libs breaking stuff even in patch releases.

It happens and it's far from particular to Julia. I just make sure to test stuff property before going to production.


Pythons breakages have also been security centric. I'm not a big fan of it, but yea Julia isn't 30 yrs old and it's done things like this every minor release without anouncement. To be clear though I'm not talking about a conscious researchable decision to drop support after 5 years. I'm talking about bugs that pop up over night. Anyway has Julia ever been audited for security? Most of the netstack was written by a single person... Makes ya wonder...


Why do you update a programming language version in production with deadlines in the first place?


Because it has a new feature that's save the day(if it worked), fixes bugs in a janky patched Julia version, and production in the data science world looks different in research and development phases then it does for software. Production in r and d can simply be, boss wants to see the pros and cons next week with a successful run. Only with Julia have I run into these kinds of predicaments


Other languages and libraries break as well so that's just bad practice.


Not as frequently. Having a broken release for a programming language is also bad practice.


Look at the backlog of issues in their GitHub for backports. You'll see detailed lists of reported and fixed breakages. The lists aren't small... Now go on to imagine the ones people don't report and instead patch and fix them locally. We aren't even talking about Julia code fixes, often these involve the C that creates it and are a nightmare to diagnose. I can't reply to your post unfortunately the thread is too long.


This is the list of bugs fixed in the upcoming Matlab release:

https://www.mathworks.com/support/faq/pr_bugs.html

Just look at the first one: "In certain cases, pointAt method of satellite scenario Satellite class interprets Euler angle inputs in radians rather than degrees"

Your expectations are completely unrealistic.


Your entitled to your own beliefs, but Julia is more of a research project that's crowd sourcing phds to make products for them then it is a programming language. In other programming languages I've used past a 1.0 release (hint that's many) I've never seen the type of breaking bugs that occur. If Julia is this end all be all scientific computing language then I expect the CSV package not to be incompatible with Base after a minor version bump. Having to manually compile a patched fix to read csvs is not normal... Sorry, but that's never happened to me in R, python, Go, Rust, C++, Java, Scala, JavaScript, etc. If that's an unrealistic expectation then maybe Julia is just holding an incredibly low bar compared to it's competitors.


Other languages don't have that? CPython has almost 7k issues open atm. GHC has 5k. Again, FUD.


It's not about having issues. It's the type of issues and how they affect end users. There's no FUD associated with that. Just facts. Julia has been around for a decade, the FUD campaign stuff shouldn't even be a concern for the project at this point. Other languages from the same era are either gone, or are dealing with very different social deterrers to adoption.

I'm not typing these things out for FUD to slow Julia developers role or something. Actually I hope they listen and actually work to fix things that matter. I invested a lot of time and effort into the language. Currently I'm looking at it as a complete waste with mostly negative side effects to my career and where I could have invested my time. Trying to do one of two things, encourage change, get people on the fence about trying it to wait until it's actually ready.


How exactly are Julia releases broken?


Can you be more specific? As is, it's hard to follow what the issue was that you encountered.


At this point the only thing I can say is. Have you ever been a Julia user? Or are you actively developing Julia? If you are actively developing Julia, yea sure pull down the feature branch after someone's reported a bug, hot fix your toml to point too some specific version after reading ten diffs to be sure you have the right one, maybe stand up your own package server or pay 50k for one from julia computing, recompile Julia itself and everything is totally fine! If it's not no big deal just stomp over the type with your own code, yay. Maybe that's even part of your job. If you're a user expecting to do development and receive updates on packages or language, all I'll say is best of luck.


I run in production several Julia projects and I honestly can't say I know what you are referring to.

I do use some libraries through PyCall. This is a relatively frequent source of trouble, with libraries breaking because of 3rd party dependencies, behaviour changing, or even somehow installation suddenly stops working. This all happens in the python side, to the point that I am currently replacing all the libraries even if it means porting some of them or even a loss of functionality.


You can use CondaPkg.jl (https://github.com/cjdoris/CondaPkg.jl) to setup Python dependencies with version control. I haven't played with it too much but it seemed to work out for what I tried. Indeed the reason why I haven't had too many test cases is because in SciML we removed all Python dependencies since they were the main source of instability. PyDSTool.jl, FEniCS.jl, SymEngine.jl (through ParameterizedFunctions.jl and ModelingToolkit.jl) were the biggest development burdens because of the Python deps changing function names all of the time until they were deprecated and replaced by pure Julia packages, so I definitely know the pain.


Is it fair to say that it is your career to rewrite code in Julia to write academic papers about julia?


Most of my time isn't academic or writing code. Most of it is focused on enhancing algorithms to achieve what's required for new methods to work deployed in production environments such as the Pumas clinical pharmacology work, and now working on what's required to get scientific machine learning point-and-click GUI ready. I always leave a little bit of time in mornings and weekends though for some coding, and a little bit a day goes a long way after years.

I stay away from code rewrites like the plague, which is why SciML maintains so many wrapper packages. It's less work and if there's nothing new then rewriting is thankless work. Unless we have some kind of angle for something new in the algorithm, we just recommend someone use the same old wrappers (for instance, IDA with fully-implicit DAEs). We won't beat the C/Fortran code without something new. However in most areas there's lots of interesting research to be done. I could "sabbatical as an academic" and spend the next 3 years just improving explicit Runge-Kutta methods: those for example are probably still 4x away or so from what I think is theoretically possible, but that's not going to happen any time soon since we have some real applications to focus on.

Those package deprecations mentioned above were deprecated by community efforts (BifurcationKit.jl replaced PyDSTool wrappers and was done by Romain Veltz, FEniCS wrappers were largely replaced by Gridap, and SymEngine was replaced by Symbolics which is largely the work of one of the Julia Lab PhD students Shashi). Even those had a largely new element to them though, with BifurcationKit focusing a lot on recent algorithms that other bifurcation software don't have (like the deflated Newton methods), and Shashi's Symbolics.jl work focusing on generality of term rewriting systems to allow for alternative algebras.


Think you've dodged the point entirely by fixating on a word rather then the sentiment. Isn't it in your professional interest to endlessly promote Julia at all costs. To find tools in other languages and make sure they are available in Julia, etc?

That's basically my point. Readers should be aware of who is making suggestions and why they might be making them. You have a clear status and financial drivers to say "only use Julia, it's ready for anything, it's fine" while not saying any of the flaws other then "it's fixed if you compile the branch of the grad student I asked to fix it's PR".

Also the type of work you do is incredibly rare. Most people considering trying to use Julia for their day job don't have the resources you have nor the needs or desires you have. In most cases getting one of the most foremost scientific computing in Julia experts take on what tools they should use really doesn't map well to practical users.


So much for solving the 2 language problem. Everytime I ended up in a similar situation the Julia code was rewritten either by me or someone else in c++ for practical/performance reasons. Hope you consider the same. The libraries for doing mathematics in C++ are surprisingly good and you might be surprised to find how easy parallel processing and fine tuning is in c++. In some ways it's less cognitive load than Julia because it's more explicit...


My problem is not with Julia, my problem is with the Python ecosystem and its dependency management. Not sure how you can read what I wrote and reach that conclusion.


We run Julia code in production. These are compute-heavy services called by other parts of the app. Version upgrades are normally easy. Upgrade, run the unit tests, build the Docker images and deploy to K8s. Maybe we've been lucky, but since 1.3 we've never had big issues with version upgrades.


That's wild, I've seen huge performance regressions from changes to Base, among commands being dropped, irreconcilable breakages with basics of the ecosystem, spurious bugs in the networking libraries that are nearly impossible to chase down, and the menagerie of compatibility with essential packages break. I stopped caring around 1.7 when I realized this was the norm and it wasn't going to change. You must be honed in on some specific packages with a lot of custom code.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: