> At first you run into all the usual tool roughness that you'd expect from new experimental features. For example, GCC generates dependency files that don't work with Ninja.
To sum the link behind that statement up: The default output format used by the GNU Compiler is for GNU Make, a ninja compatible format can be generated but is not the default. The blog author considers this default broken, GNU developers consider this behavior correct.
> a ninja compatible format can be generated but is not the default
The suggested flag was not for a "ninja compatible format", but to exclude things from the makefile snippet that ninja doesn't understand (yet). It would exclude necessary information for modules to function.
Problem is, the new syntax used by the compiler's -MF output violates current-day expectations by both compilers and build tools as to what those snippets contain, and it'll be a while to sort out what the acceptable syntax would be. As noted, Clang doesn't use this file to propagate the information at all.
> gcc (and other compilers like clang) support emitting dependency information in the syntax of a Makefile.
But didn't quite make it to the second paragraph:
> (Ninja only supports the limited subset of the Makefile syntax emitted by compilers.)
Obviously Ninja can't parse and execute full Makefile syntax because then it would just be Make. But the actual information in depfiles doesn't require the full Makefile syntax; only a subset of it. And compilers output that subset, so Ninja can read it.
The issue is it's a de facto subset - a "depfile syntax" that exists by convention but the GNU authors have broken this format. As I said, unless there's a good reason to break it, that's just shitty behaviour.
I already knew that Ninja only supports a limited subset and that the issue is the compiler now going beyond that limited subset.
My point is that I wouldn't jump to conclusions as to why they did that. I would dig further to see what the reasons were (which is not a simple google search) and what Ninja developers could do about that.
Another important point is that they didn't break Ninja compatibility in general, but only in the specific case of C++ modules; and they also didn't break compatibility with non-GNU implementations of Make.
> But the actual information in depfiles doesn't require the full Makefile syntax; only a subset of it.
That seems to be the problem, the information introduced for the module support[1] apparently cannot be handled with just the dependency lists alone and the changes suggested in the ticket would still leave the additional CXX_IMPORTS += M1.c++m syntax. One of the commenters on the ticket also wrote a specification for a json based dependency format[2].
How is it that D and Objective-C++ came up with modules that work and can be used many years ago, yet C++ with much greater stakes fumbled it on almost every level?
Exactly because of much greater stakes. A lot of heavyweights involved, and nobody agrees to yield. The result is a compromise, aka a solution which is unsatisfactory to all parties to an equal degree, as they say.
Best designs are produced by small, tightly-knot, often one-person design teams. Examples: Unix, Clojure, SQLite, Boeing-747, Westminster Palace in London.
Sometimes a small team with a cohesive vision keeps on attracting like-minded people or matching ideas, and the project grows with contributions from a large number of people. The key part of every such success is the vetting process. Examples: Python, Ruby, FreeBSD and OpenBSD.
Worst designs with most glaring, painful, expensive, even tragic shortcomings are produced by large committees of very important people from huge, very (self-)important companies, agencies, departments, etc. Each of them has an axe to grind or, worse, a pet peeve. Examples: PL/I, Algol-68, the Space Shuttle (killed two crews because the escape system was removed from the quite sane initial design); to a lesser degree, it's also HTML, CSS, and, well, C++ to a large degree :( The disease has a name [1].
Sometimes a relatively small and cohesive team in a large honking corporation produces a nice, cohesive design, like this happened to Typescript, certain better parts of CSS, and some of the better parts of C++. This may sometimes make a false impression that "design by committee" sometimes works.
The original Unix is also about 5 people, with Ken Thompson being largely credited with the general design.
Of course, later the decoherence increased; I'd say that the adoption of Berkeley Sockets is one of the inflection points where severely different architectural principles came into play.
Apparently Joan Mitchell and Bill Pennebaker were the key driving force behind the JPEG format [1], Ms Mitchell having already invented the widely deployed fax compression algorithm, and a few related things.
I think the reason is more that the goals of the C committee align better with the goals of C programmers (which is a language that's a tool and not a playground for language designers).
Unlike C++, C also remained a simple language (which is definitely a side effect of the "conservatism" of the C committee).
That's what he means; like the puddle which marvelously fits its own hole, anyone for whom C in all of its aspects is not very close to optimal has long ago moved to another language.
WG14 has only "worked" fairly well insofar as it did absolutely nothing but file the serial numbers off of things standardized in C++ and release them. The biggest C-original features (type generic macros, annex K) are all gigantic boondoggles. Even many of the features that were lifted from C++ were poorly thought out (e.g. compatibility between <stdatomic.h> and <atomic>).
The biggest C-original feature since C89 is easily designated-initialization in C99, and WG21 couldn't even get that simple thing right when they tried to copy it into C++20 two decades later (even though Clang had a working implementation in C++ mode since forever which just had to be copied by the standard).
After a decade of using obj-c modules I'm still unclear on what the benefit of them is other than the Swift bridging aspects of it. There isn't really any observable difference in either compile-time performance or behavior from turning them on and off other than that a lot of the rules around inclusions are much stricter with modules enabled. The documentation claims that they should improve build performance, but AFAICT that isn't actually true or the difference is so small as to be irrelevant.
It's somewhat true and it should also improve debugger performance, but it takes a very long time to get to the point where that can be turned on and anyone sees it. (keyword is "explicit modules")
Is easy, you don't have the same amount of courdination problem, most of what's wrong in software today have in big part problems whit two things legacy and coordination problems, need back compatibility means that useful solution aren't posible, soy then you need to compromise and then compromise again because some users cant use you solution, so now you have solution who don't satisfied anyone, sum that whit natural misalignment of any organization and is obviously that focus sitmrs whit few stakeholders and willingness and capacity to break change can do a better job, this ins't particular problem of programming.
There is no backwards compatibility here to speak of: C++ modules are a new feature introduced by the C++20 standard. There was absolutely no reason to make a mess out of them.
Maybe we need to interview a C++ compiler engineer, for microsoft, GCC, clang, etc, because C++ is a large language that carries its compilation model from C, and I don't think making modules happen, or implementing them, is a trivial task.
I don't think they intended to make a mess, I think that C++ compilers are just massive things with massive codebases, and several different companies who have to agree on one language.
What’s upsetting is that the feedback has been provided before that the standards body and compiler devs aren’t communicating with build systems folks when modules were first being standardized. Looks like not a lot has changed.
I don't know of anyone happy with the C++ standard committee.
Some parts of the new std features were copied from Python, which is good, a semantic and naming a lot of people already know. Yet some weirdo decided to half copy the naming scheme of APL and we end up with functions named "iota". Seriously. What. The.
Don't get me started on features added in C++17 and deprecated 2 versions later.
Don't ask also why they decided to rename universal references by forwarding references while the former makes sense and the latter none, just because they were pissed to not have named it first.
All I want now, for the future of C++, is a Circle-like approach. I want to be able to selectively disable and replace legacy behavior and syntax, per file, so that at last we can have the language evolve.
I think it has to be more of a compiler driven movement. GCC and clang have to be the ones that accept to implement the standard of an alternative committee. Now I get this alternative committee doesn't exist yet, and there is no agreement of what the future could be (cpp2, cpp front, circle, carbon..) and it's a huge investment.
But developers will follow compiler. I sure would use circle if it was not a proprietary compiler.
Except the C++ compiler world is much bigger than two compilers.
clang is even lagging behind in ISO vs GCC, because most compiler vendors that fork it, mostly care about the backend, while two former major contributors are now are focused on their own LLVM based languages.
Which is why, those that care about portable code can at best use C++14 or C++17, while many libraries still target C++11.
So how do expect those two compilers to set the direction C++ should be?
Huh, I mean yeah gcc, clang and msvc, for desktop, servers and GP embedded hardware. That's like 99.99% of the compiling that most likely happen in the wild. Far from tunnel vision.
It's because they're trying to make a module system that is compatible with decades of development based around `#include`. It's nowhere near as simple as making a new language with modules.
I almost feel like they should give up. IMO Rust and Zig have made C/C++ a legacy language. Except in some specific areas where Rust doesn't have mature libraries (GUIs & games mostly) you'd be silly to start a new project in C++.
Migrating existing C++ projects to modules likely isn't worth the enormous effort, and the days of starting new C++ projects are numbered... so why bother?
There is still an unfathomably huge landscape of problems where C++ is the more mature, “right” choice over rust for new projects. Just from my (extremely limited and narrow) personal experience:
- scientific/numerical computing (super mature)
- domain-specific high performance computing (C++ is for better or worse extremely flexible)
- weird CUDA use cases (feels very native in C++)
- game dev as you mention
- probably much much more
C++ to me feels more like a meta-language where you go off to build your own kind of high performance DSL. Rust is much more rigid, which is probably a good thing for many “standard” use cases. But I’m sure there will always be a very very long tail of these niche projects where the flexibility comes in handy.
There are tons of places where Rust and Zig don't have a solution, or are not part of an industry standard that builds on top of C and C++, like GPGPU, Khronos standards, console SDKs, ...
Additionally while they keep being built on top of C++ compiler infrastructure, C++ isn't going away.
Sure now that is the case. But which do you think will happen first - C++ will get working modules or people just give up on C++? Definitely not clear cut!
> In other words in order to be able to compile this source file you first need to parse the source file and all included sources until you hit the export declaration and then throw away the result.
I might be completely wrong here, but reading this reminded me of barrel files [1] in JS/TS projects, which are an absolute nightmare. I put together a little visualization a while back that demonstrated how much extra work is required to resolve the module graph if you forward exports, and it is orders of magnitude more than just importing something directly from the file where it is exported. If this is also the case for C++ modules, I'd like to buy the author a beer or six.
Unfortunately, there might be too much IP in the code (it was for a team presentation at a company I contract for). I could probably scrub that out and put it up on GitHub, though.
Do we even have a second C++20-standard-compliant module implementation? Last I checked, it was just MSVC -- GCC and Clang were like 90% but missing a few features.
If by fixed you mean "elaborate measures for specificity and saliency but still the same order of undefined behavior & bugbears" then yeah? It's gotten a lot more... more.
The author says Fortran modules are easier for build systems to handle:
'A Ninja file is static, thus we need to know a) the compilation arguments to use and b) interdependencies between files. The latter can't be done for things like Fortran or C++ modules. This is why Ninja has a functionality called dyndeps. It is a way to call an external program that scans the sources to be built and then writes a simple Makefile-esque snippet describing which order things should be built in. This is a simple and understandable step that works nicely for Fortran modules but is by itself insufficient for C++ modules.'
...
'Let's start by listing the requirements [for modules]:
All command lines used must be knowable a priori, that is, without scanning the contents of source files
Any compiler may choose to name its module files however it wants, but said mapping must be knowable just from the compiler name and version, i.o.w. it has to be documented
Changing source files must not cause, by itself, a reconfiguration, only a rescan for deps followed by a build of the changed files
The developer must not need to tell which sources are which module types in the build system, it is to be deduced automatically without needing to scan the contents of source files (i.e. by using the proper file extension)
Module files are per-compiler and per-version and only make sense within the build tree of a single project
If module files are to be installed, those are defined in a way that does not affect source files with modules that do not need installing.
Fortran modules are great for developers but a bit of a pain when setting up a build system (even though it’s quite straightforward now, it still requires a first pass with something like makedepf90 or one of its many equivalents). I find it quite a feat that C++ modules, while benefitting from the experience of many languages, ended up so broken.
Also, I never really liked cmake but this looks really bad.
If your language has modules, like in the syntax and semantics, and you're still requiring external tools to hunt down dependencies, your module system must somehow be a failure.
The idea behind modules is that since they are part of the language, the compiler for the language understands modules and follows the dependencies. All module-related tooling, like incremental builds, is hooked through the compiler. You don't use make or external dependency generators.
If someone does need a dependency generator for whatever reason, the compiler has to provide that function. You give the compiler the proper option to spit out dependencies, point it at the root module of the program, and let it do its thing.
> If your language has modules, like in the syntax and semantics, and you're still requiring external tools to hunt down dependencies, your module system must somehow be a failure.
No argument from me :)
> The idea behind modules is that since they are part of the language, the compiler for the language understands modules and follows the dependencies. All module-related tooling, like incremental builds, is hooked through the compiler.
I am not sure I follow. The compiler still compiles one file each time it is run, so we need a first pass to determine the dependency tree and what can be compiled in which order, right? Even if that first pass uses the compiler itself (doing only enough parsing to see what is needed).
> You give the compiler the proper option to spit out dependencies, point it at the root module of the program, and let it do its thing.
Nowadays, gfortran can generate Makefile fragments with the dependencies, but it was not always the case. This is why there are a handful of small tools to do it and makedepf90 is one of them (it’s a small Fortran program with a rudimentary parser). There are also other compilers that don’t do it properly.
To sum the link behind that statement up: The default output format used by the GNU Compiler is for GNU Make, a ninja compatible format can be generated but is not the default. The blog author considers this default broken, GNU developers consider this behavior correct.