As a newcomer it’s a fantastic language but the tooling and the ecosystem story is really lacking. I really think the language needs more hype in order to get more people to do things for it. The versioning story of dependencies is almost inexistant, the interaction between the package manager (opam) and buil tool (dune) is filled with issues, there are no conventions in the way you set up a project (you can put your files in any folder), stdlib vs base? Poor documentation, poor compiler errors, poor managements for patches (opam pin) and different versions of the compiler (opam switch creates a local repo in all your projects), etc.
Coincidentally I’ve been writing up an OCamlbyexample page that hopefully I can post here at some point to motivate people to learn the language:)
I'm an actual OCaml programmer, almost daily, and I don't understand this objection. We use autotools and make as the build tool (but you could use ninja/meson, cmake or whatever), and don't bother with opam/dune. We properly package over 100 OCaml packages in Fedora. For anything else not covered by a package we mostly call into C libraries, which is very easy to do from OCaml.
I guess it depends on exactly what you're trying to do, but we've developed a large amount of complex software in OCaml this way.
You roll your own build system instead of using the built-in tools and you don't understand why people wouldn't want to do that? This isn't a typical situation in Rust or even Haskell these days. Come on.
Rarely are people arguing that a situation like this makes writing software impossible. The question is whether it adds friction, and in adding friction impedes adoption. I think it clearly does, and that tooling is one of the highest-impact features a language can prioritize to improve adoption. If the story for building and packaging OCaml software is "go make yourself proficient in the C build tooling ecosystem first," of course that makes it harder to adopt (and you're not the first person in the community to give this answer, either). This hostility to beginners is one of the major things that irks me about the OCaml community, and OCaml is probably my favorite language.
Is it "being hostile to beginners" to encourage them to learn simple universal tools to build any software in any environment?
When i was a beginner, trying a new language was actually easier: you just installed a compiler. Now you need a "platform", for some even a special dedicated machine.
I wonder who is benefiting this sillo-ing, but I doubt that's the beginners.
I'm not talking about beginning programmers, I'm talking about people who are new to your language community. You can still just install and use ocamlc/ocamlopt, or rustc, or ghc, but the bar has been raised. People picking up a new language today for production use expect it to have a cohesive story for builds, packaging, tooling, and testing out of the box.
>I wonder who is benefiting this sillo-ing, but I doubt that's the beginners.
Everyone benefits from the better tooling we have now. The siloing is a problem that still has not seen a satisfactory solution. Regression in user experience is not acceptable.
It baffles me how much younger devs seam to be unable or unwilling to consider anything else than "communities". I guess the latest 20 years of research in web marketing did that to us.
Being part of a "language community" is not the only way to be a programmer. Not knowing your way outside of the opinionated built tool black box is indeed quite frankly the hallmark of being a beginner, if not a tech illiterate, in my book.
When I originally asked what was the point of Dune, the answer I received from the people pushing that project at that time was that I shouldn't worry, it was just a tool to speed up getting a workable workflow when teaching 1st grade students. It made sense then, I though at some point beginners would naturally outgrow that walking aid. But look at what happened instead!
Mind you, today's experience with ocamlc/ocamlopt for the seasoned programmer is not unchanged either. Because of that new "community first" principle, all the tools tend to work only withing the ecosystem at the expense of cooperating with the larger distribution. Again, younger devs or devs used to "lesser operating systems" never experienced life under a proper system-wide software distribution so they have no idea what they are missing. Of course opam looks great once Debian is broken (and until, let's hope, guix or nix takes over). An exemple of such a regression: querying the type of an ocaml expression from any editor used to be trivial from the easily parsable annotation file producer by the compiler. It's now been replaced by a memory dump that's easier for merlin. And merlin will not even be able to answer that simple query unless you provide it with a full blown configuration. Good luck setting that up without the assistance of the dune built tool. Good luck setting that up if your project involves anything else than plain and simple ocaml.
Ach, I probably start to sound unfair. Indeed this tendency toward infantilism is in no way specific to ocaml, probably ocaml has been resisting it for longer even. And it's not specific to programming languages either, nor even to technology. Pardon my rant, grumpy old man can still feel passionate about computers at times; rest assured he is being transferred to another trade :)
You’re just assuming a lot of stuff I didn’t say. I have a lot of respect for your perspective on software and always love hearing from more experienced programmers. I think universal tools are great for a lot of stuff, but they’re just not as productive as cargo and stack are. Have you tried really getting into one of these? Having package management, toolchain management, builds, testing, formatting, linting, and IDE support out of the box with one tool is really cool, and it’s not something that the distribution-centric tools ever provided.
I will say that adapting to change is part of the human experience and especially part of the programmer experience. I’m 22, but I’m almost entirely self taught as a developer and have only a year of college. I’ve been reading older programmers (and older people) lamenting the new way of things for at least ten years already, it seems to be a constant. What you now call infantilism will be the old way 30 years from now, and the kids writing software then will be doing stuff that confuses me, I’m sure. But this is how progress happens, and change is okay.
> You’re just assuming a lot of stuff I didn’t say.
That's because I took that opportunity to rant more largely about the whole industry.
I frequently ask myself what's the likelihood that this perception that the computing industry is regressing, and that is shared by many of the older devs who were passionate about it, would be just caused by normal grumpiness and confirmation bias. Eventually, I believe tech evolution is more dynamic than progress/regress.
I've witnessed the creation and then the demise of personal computing, unrestricted computer networks, free and open software distributions, unbiased search engines; I hope I will not be around when the Linux kernel momentum is lost and personal general purpose computers become once again inaccessible. There were of course already a lot of deficiencies when I started my career ; the big "Software Crisis" was already well documented, some big corporations were slowing down innovation as hard as they could, and surely many established programmers routinely wrote miles of buggy software on inefficient hardware. But there was a growing, vivid shiny trend, easy to spot, in all this gloominess: a Unix revival on micro computers led by young engineers who just wanted to do the right things. Ok, that momentum is gone now and I'm looking for the next big wave that would push us forward. Meanwhile: business, politics, conventions, ignorance and laziness are eroding that culture.
This is how I picture things, at least; in this picture progress is not automatic, nor is regress.
Please do not believe progress is automatic, or you won't fight for it.
At worse, if that is wrong and progress is indeed automatic, what's the loss?
I don't believe progress is automatic, that's why I like hearing perspectives from people who have been around longer. When I hear the narrative from lots of older people that everything has gotten worse since their youth though, I think you can understand my skepticism. Especially given as your opinions are not universal among people who have a lot of experience.
Anybody today can, in seconds, download dozens of production-ready language runtimes and get started writing programs with a great IDE experience, for free! No cost at all. And this is now a fundamental assumption of software development.
I don't take it for granted because I have some idea of how far we've come, but I read people like you complaining about not being able to work with new build tools and I'll be honest, I assume that you've been left behind technologically and haven't kept up. I'm sure that's unfair, but you don't give these tools credit for their upsides (using new languages is much easier than it used to be, and development with them also scales much better), and you still haven't really explained the downsides fully. Even the C and C++ community is slowly moving in the direction of package managers and integrated tooling.
Sure, when I was younger I remember trying every single package from my distro or that was featured in slashdot or that appeared on freshmeat.net etc. These days my default attitude toward a new piece of tech is a shrug, and the list of techs that I (wilingfully or not) do not learn and skip over is becoming larger and larger with time. I do not believe that's because I'm becoming lazy though; fatigue sure is a thing, but we're living at a time when our universe is expanding faster than the speed of learning anyway. And I believe you too are staying ignorant of most of novelties.
Do I stick to my tools longer than necessary before acknowledging true progress?
Possibly, but not always. I've been pushing ruby over perl/php/python, ngnx over apache, I adopted systemd quite early for some of its practical merits, I pushed for containerd over docker, for nix then guix over Debian, and PL wise I've enthusiastically explored mercury, ATS, and rust way before it was a thing (then decided against it). So although it's true that I would not feel confident in a conversation with young JS programmers I could still name a long list of interesting new techs they have never heard about! :)
One of the hardest thing in a software dev's job used to be to learn to say "no" to product-designers and management. Nowadays it's to say "no" to shiny new techs. Many times this pays off, since most of tech novelties shine only for a brief moment before being superseded by another. The price to pay is to arrive a bit late to the party from time to time. One have to be very passionate and picky to not end up stranded in an isolated ecosystem.
You mentioned IDEs many time. Beware that they are often times such isolated ecosystems. You would not believe how strongly java devs thought no one would ever need to venture outside of Netbean... No I mean Eclipse. No, VScode. Meanwhile, I'm still wondering what problem those are trying to solve; do I have a problem I haven't diagnosed? Must not be the speed of writing or navigating code though, given I'm usually amongst the fastest around.
In programming as well as in real life, things own you as much as you own them.
I can tell you in my case, older folks who still program can be VERY picky on what new tools they elect to invest time in learning. When we elect to "keep up", we do it in the general case and I'll tell you why from my point of view, there have been times in my life when I've chosen to dive into details of a tool, that I would then be disappointed in what I discover and now this time is spent, I cannot get this time back. When we're young, time is cheap.
Look, "keeping up in detail" is the domain for youth because youth has the luxury of the spare time to do it.
Besides, being left behind is actually the destination for all of us simply as the result of our own mortality. I'm suspicious that you might think that the fact some folks "haven't kept up" indicates an error on their part rather than a very valid choice.
heh and in my case I'm just not that good a programmer so theres that.
These language-specific toolchains are often not better. They're complex, opaque, stuff things in places in my home directory, don't work together, and don't work with system packages -- golang is actively hostile to being properly packaged, making it difficult to keep track of what's installed and security updates.
Now if you're saying that the autotools/make/cmake/meson tooling is hard, I sort of agree, but many people are familiar with C build tools already. I don't see how it's easier or harder than learning language-specific tools.
>Now if you're saying that the autotools/make/cmake/meson tooling is hard, I sort of agree
That's not really the main problem to me but it is a problem.
>many people are familiar with C build tools already
Maybe if they're C++ programmers. I don't know anyone outside of the C++ community who has ever used anything aside from Makefiles and language-specific tooling.
>I don't see how it's easier or harder than learning language-specific tools.
The language-specific tools are typically very well integrated with the default testing, packaging, and editor tools ecosystem. The C build tools are not going to be, at least not for OCaml. Language-specific tools show users the best experience a language can offer. More advanced users can always choose to use something else, but the advantages of the whole community using the same tool are hard to beat as well.
I guess we know different developers. I work with Linux / C programmers, and everyone is familiar with the basic C build tools. C tools also deal with testing, and there's a very clear and well-travelled path from ./configure && make && make check && make DESTDIR=.. install to RPMs and Debian packaging.
That's the process for building and installing packages though. I'm talking about writing new ones, on a system with nothing else installed.
I'm omnivorous but I get paid for web development and that's what most people I know are most interested in. And like I said, they understand Makefiles and make, but autotools and Meson are not "basic C build tools," they're both extremely complex and relatively niche. I'm not saying you shouldn't use the tools you use, but you should understand that they are not popular or well-understood outside of your niche of Linux programming with C. I'm sure these programmers could learn how to use them, but most are going to choose not to if it's the easiest way to use the language. They will pick something else.
If you browse job listings, you might be surprised to find that the vast majority of jobs these days are not related to C or Linux development.
Companies are mostly building web and mobile applications, along with the corresponding web servers on the backend. JavaScript, Swift, Go, Java, Python - these are the kinds of languages developers are working with, and none of those languages normally are used with C build tools.
I've done a few small projects in OCaml. It's a brilliant language that I deeply enjoy using. But the situation for newcomers and those who don't have established patterns is bad. It's hard to do anything nontrivial from scratch, and the community doesn't seem to have any consensus around dependency management, builds, project structure, and so on. Make and autotools may work for you, but it's not like your setup is on the ocaml.org homepage, reproducible by an application developer in five lines of shell commands.
It's common now in some language ecosystems for the default tooling to just generate all this stuff for you, and you move on to writing code. So if you're e.g. a JavaScript developer who is interested in functional programming, and you're used to npm or whatever, it's a difficult start.
The commenter I'm replying to uses an entirely different toolchain, and the frequently-recommended Cornell book (https://www.cs.cornell.edu/courses/cs3110/2021sp/textbook/) uses ocamlbuild and utop and makes no mention of Dune. If you (understandably) happen to land in the official OCaml manual at https://ocaml.org/manual/index.html instead of the page you linked to (which I can't find from ocaml.org, by the way, but you're right that the ocaml.org tutorials do mention Dune and opam), then you don't see Dune or opam mentioned at all. I get what you're saying, but I don't think you could disagree with the premise that there are parts of the OCaml ecosystem that a newbie would land in and be confused or misled by here.
That's fine, but if you go to https://ocaml.org/ and click the big 'Install OCaml' CTA at the top of the page, it takes you through installing opam, dune, and ocaml-lsp-server. And as per the latest OCaml Survey, it looks like the majority of the community are using opam and dune: https://docs.google.com/forms/d/1OZV7WCprDnouU-rIEuw-1lDTeXr...
Do some of the doc pages need to be updated? Yes. Do some universities take a long time to update their course materials? Yes. Does this mean there's no conensus? No.
There appears to be a rough consensus but it seems to have emerged relatively recently. I learned OCaml about two or three years ago and at the time the best tooling was Tuareg and Merlin in Emacs. Fortunately for me I love Emacs, but I found the Dune documentation poor and overall I got the sense that the OCaml production ecosystem (vs academic) is largely dictated by Jane Street these days, but the community is kind of quiet about the extent to which Core is the "real" standard library and the default stdlib is an academic curiosity. For one thing, the community seemed to be in a transition period between ocamlbuild and Dune, as the official resources recommended ocamlbuild and the community said everyone uses dune.
Yes, this is changing, and these changes have been ongoing since about 2013 when opam was created. But it's true that they're accelerating. I think it's reaching critical mass. so yes a few years ago I would say your points would be valid, but today they are less so.
Dune itself is only few years old. And there has been quite a lot of work on consolidating ocaml.org as a clear entry point for newcomers. But that is still an on-going work.
The C-ecosystem build tools you refer to are pretty far behind those of pretty much every other langauge. And most developers don't want to wait for their OS to package it to start using the new version of a library (not to mention this style of packaging isn't typically available on windows/macos).
Really? You can put your opam package files and dune files wherever you want and it’ll automagically work. Submodules can be defined far away from the library and it’s fine. If your opam packages are empty it’s fine, dune doesn’t care about versions of packages anyway. If you want to vendor/patch a library, just drop it anywhere in your project. Yet dune seems to be unaware or executables and libraries when you want to run or build something (you need to pass paths).
That's mostly because dune is 'composeable' so it accepts project components anywhere in the tree. But the components themselves are laid out in a certain way:
- A dune file in a specific directory makes that directory a component
- An .ml file in the directory with the same name as the component (in the dune file) becomes the main module of the component
- Any submodules aliased inside the main module are properly wrapped and namespaced.
Sure, but Cargo and go modules/vendors are composable too but there’s still a forced convention that doesn’t allow you to place your files wherever you want.
Oh and, the way name and public_name works is also not great. Basically you never know what is being importer or used because it ultimately depends not on how the folder is named but on either name or public_name in a dune file. For example in rust you can import what’s in folder a/b by doing a::b but in OCaml you import a.b or a_b it it’s what set as name or public_name.
Coincidentally I’ve been writing up an OCamlbyexample page that hopefully I can post here at some point to motivate people to learn the language:)