Reading these stories one keep seeing the same pattern. GCC vs. EGCS, just like the Emacs war. The problem are always an excessive centralisation of stewardship... which is kind of ironic coming from the guy who most strongly supported free software. On the other hand the fact that they could be forked and then merged shows some of the advantages of FOSS.
Maybe it will be similar with LLVM. Back in the days Apple suggested to put a gcc front end to LLVM, but the GCC people were strongly against it. They seemed to have a really dislike of LLVM, or its licence, or the company behind it.
Eventually Apple (and the rest of the LLVM community) continued on their own. If GCC one day will become irrelevant, I think it's got only itself to blame, and a certain type of elitist arrogance. I wish (for them and the FOSS community in general) that the successes of LLVM will help them clean their house and keep going for decades and decades.
I think the dislike of LLVM stems mostly from the company behind it. As you may know, Apple contributed a fair amount of code to GCC, but they also kept a lot of code for themselves. (They did, of course, make it available per GPL, but did not try to submit it upstream.) There have also been problems in the way Apple contributed code; see the mailing list archives for details.
I think there's also some dislike of the project itself, or perhaps the people behind it. But that's another discussion.
> The problem are always an excessive centralisation of stewardship... which is kind of ironic coming from the guy who most strongly supported free software.
I don't think it's so much centralization of stewardship for technical decisions, but stewardship of the rights to the code; i.e. you need to sign copyright papers before committing non-trivial patches. And I can assure you that the FSF sees doing so as strongly supporting the cause of free software. There are obvious counterexamples (the Linux kernel's more lax Signed-off-by policy comes to mind), but there are reasons why this centralization is done.
I know this centralization causes problems. Getting assignment papers submitted if you are at a university is a pain; your employer may not be keen on signing them either. It'd be nice to see the process streamlined, or even banished entirely, but I doubt that's going to happen.
> I think it's got only itself to blame, and a certain type of elitist arrogance.
I'm curious where you think this "elitist arrogance" comes from or even what sort of behaviors you're referring to. Can you elaborate?
Wasn't one of the problem with Apple and GCC that Apple had to hack the GCC to integrate it with the rest of its tools (which obviously the GCC people couldn't care less about)?
When it comes to the topic of stewardship the situation is less clear cut from my point of view. I think there's a (very human) feeling of "I've created this toy so I get to decide". In part it's a good attitude: if you managed to develop it this far there are good chances you'll be able to develop it further. But sometimes it makes people a bit blind when something better comes along.
In that sense using technological reasons, or the cause of free software, seem to be more of a way to (maybe not on purpose) push away whoever wants to change the way things are run. But that's purely a gut feeling.
Incidentally that's what I tried to convey with "elitist arrogance". This kind of "we know best" and "who are you? what are your credentials?" attitudes. Though I am not surprised it only made thinks more confused to a reader. Sorry about that. :)
I don't know the full story about Apple and GCC; I haven't read all of the mailing list traffic during the relevant period, nor was I around to hear about offline conversations. Certainly I can see Apple modifying bits of GCC to make it more suitable for IDE integration and upstream not caring about that. Apple did have ways of getting such changes in, though; it's possible they didn't care enough to push them upstream.
As for "elitist arrogance", I think what you are getting at is that you see the FSF's policies as erecting a moat and a wall around GCC and telling the rabble to stay out while the nobility quietly hack away at their desks. If I have understood you correctly, then I see the argument, but I do not agree with it. As long as GCC is an FSF project, that's just the way things are going to be. I don't think the FSF has set out with the intent to be "elitist"; it's unfortunate if that's the way the policy is perceived. Feel free to correct me; it's possible that I've misunderstood you.
The FSF does not, on the whole, dictate the technical direction of the project. So the goodness or badness of technical changes and any "elitism" associated with acceptance or rejection of such changes is another issue.
Perhaps things will change if GCC ever disassociated itself from the FSF: another egcs-style fork or something more dramatic. There have been small rumblings of such a change, but I think such an event is quite unlikely.
Please understand that I don't think the FSF ever set out with the intent of being "elitist", and I appreciate what they are trying to do. Although I often do not agree with the methods, I am happy they exist.
I like your example of the moat. I think though that nobility is not very much defined in your metaphor. After some thought I think there is a strong sense of community maybe, where those who've worked there the longest are those who become the "nobility". And people do not want a "foreigner" to come and change their ways, even though he may back his ideas with facts and good arguments.
In a way, a method that made sense once, is used for such a long time that at some point it becomes more of an "ideology", like a nationalistic feeling.
I think it's a lot less drama-inducing to consider any Apple contribution to open-source as a fork of the original project. The KHTML people were just as annoyed that Apple were bad open-source citizens, but there's nothing in any FLOSS license that requires you to play nice with the existing maintainers of any project if you don't want to.
I wasn't thinking about the ObjC frontend; the ObjC frontend, AIUI, was contributed by NeXT before NeXT became a part of Apple.
If you go grab the GCC tarball Apple provides, it has many, many changes from the GCC 4.2 sourcebase. Lots of those changes have not been submitted back upstream. The code is available, granted, but it's not available in a way that's helpful to anybody but Apple.
So? They complied with the GPL. Why is it that people also expect companies to push code changes back upstream, maintain said parts of code upstream and properly document said patches?
If upstream wants patches so badly they can download the tar-ball provided by Apple, diff the two source trees and document, and submit those patches to the upstream. It is not the companies responsibility to do so. The GPL only states that it MUST make the source code and its changes available.
While LLVM isn't a fork and doesn't quite fit the model of the gnumacs/xemacs, gcc/egcs, glibc/linuxlibc &c. from years gone by, I do hope the new competition in the free compiler space will spur improvement and innovation in both projects. I would almost expect not only the state of free compilers, but the health of GCC itself, to be better with a viable competitor in LLVM and clang than without.
Seems to me (an old compiler guy in previous lives, but not necessarily up to date technically) that LLVM has really taken the momentum from the GCC project, since they're starting afresh, and generally being more approachable.
Perhaps someone with technical experience with both internals could comment?
Disclaimer up-front: my day job is hacking on GCC and related GNU pieces of infrastructure, so that colors my opinions somewhat. I speak only for myself, not for my employer.
My level of experience in both: I hacked a specialized LLVM inlining pass in graduate school. I've written two middle-end passes for GCC and tweaked GCC's inliner in similar, though less invasive, ways. I found the level of difficulty to be similar between the two projects.
Other people may have different opinions. I remember reading a claim of a graduate school project that tried for four months to get started on a middle-end optimization pass in GCC and got nowhere; after switching to LLVM, they made progress in a month or so. Personally, I think that meant they weren't trying very hard with GCC.
Given your experience with both, what would you consider the pain points of getting a grasp of each when looking to contribute? I know when I looked at GCC, the mix of custom manual memory management (gcc_free and pals), semi-automatic management via obstacks, and automatic management via garbage collection seemed baroque and frightening.
I don't have a good sense of pain points in LLVM; the inliner hacking was a while ago, I don't remember many of the details, and LLVM has surely changed quite a bit since then.
As for GCC, I think the pain points are twofold: the documentation for the middle-end is somewhat scattered. I honestly think enough information for figuring things out is present, it's just not always obvious where to look. There are lots of other passes to look at too, which can be extremely helpful. Assumptions of the interface, or side-effects, are not always stated, which can be surprising at times. The other pain point is contributing upstream: you're going to get dinged on formatting, documentation (usually just "did you do it"; the review is generally not as thorough as say, GDB's documentation review), compilation time, etc. etc.
Also, GCC's hash tables (htab_t) are a pain to use correctly.
Of course, my experience is somewhat slanted towards the middle-end; the set of pain points is somewhat different if you are working in the front-end or the back-end. And my set of pain points from the middle-end might be different if I had worked on different optimization passes. (My passes cared very little about things like aliasing, for instance.)
It's surprising to me that you mention memory management as a pain point, though I can see how the variety can be bewildering. The only distinction that really matters is between GC'd and non-GC'd memory. obstacks and alloc pools are only ways of providing specialized malloc interfaces. A useful rule of thumb is that if your data is only needed for one pass of the compiler, then you can allocate it any way you like; if the data is longer-lived than that, it needs to go in GC'd memory. I can elaborate if you'd like, but that's the basic idea.
FWIW, I agree that the whole GC system is somewhat baroque. The GC was a decent solution to an engineering problem and the whole mechanism nicely solved memory management problems and provided the basis for precompiled headers, but it causes problems in other ways nowadays and trying to get rid of it would be a huge effort.
Last section on page 3: "GCC code lacks clarity (intermediate semantics are opaque) as a matter of policy, and that this has worked against the greater design goals of GCC.. By making just building onto GCC and GPLing the code the path of least resistance, RMS hopes to motivate more people to produce free software."
Last paragraph on page 4: "GCC 4.5 was released in April 2010. A major new feature is a plug-in facility for third party code. One of the first plug-ins on offer is DragonEgg, an LLVM back end for GCC."
Summary: Competition from LLVM has finally forced gcc to create a plugin architecture. It'll make less software free, but it'll get more people using gcc.
Keep in mind that when GNU/FSF say "free software," they really mean it's the software that's free, in an "Information wants to be free" sense. The software's freedom necessarily confers some liberties to you as well, but they still feel that software whose freedom can be taken away has a lesser quality of freedom than software that is guaranteed free forever. (You'd probably say the same about people — if your freedom can easily be abridged, it's worth less than inalienable freedom.)
That's not the sense I get from their manifestos. They seem to come more from a "freedom of end users to modify/share the software they receive" perspective, not the more cyberpunk information-wants-to-be-free perspective. The reason they like copyleft is because it makes it harder for a user's freedom to modify/share any software they receive to be restricted somewhere down the line: you're guaranteeing that not only are your immediate users given that freedom, but so are any indirect users of your software, via any derivative-works path.
You slightly misread what he said. He said it would make less software free, not that it was less free. One of the reasons GCC did not have a plug-in architecture was to make it so that if you wanted to do certain things with GCC, you were forced to make your code free. Now you can do some of these things without making your code free.
People that hold on to the GPL as their saviour believe that if one can take code, change it, and NOT have to give your changes back to the community it has made the code LESS free.
They don't look at the freedoms each license gives but just their own selfishness of wanting everything shared for the supposed greater good, when those of us in the BSD community really don't worry too much since most companies put back BSD licensed source code anyway since it is in their own best interest. Giving the companies a choice is something they can't fathom.
- Wasabi Systems (which contributed back less and less, except for journaling support just before it went down).
- BSDi (contributed almost zero, until they merged with FreeBSD Mall).
- Apple, contributes non of their differentiating technology back to the BSDs.
- Practically any closed-source system used the BSD TCP/IP stack at some point, few contributed back.
I loved the BSD license. I still love the Apache 2 License (which is very BSD-like). I was a developer in the NetBSD project. But as much as I dislike(d) the GPL, the license is very well able to make users contribute back. The Linux kernel has a very healthy pool of enterprise contributors. *BSD development is very much driven by free-time contributors and very small consultant firms. And it shows. It is great that such such a small pool of contributors can build great systems. But Linux has outdone BSD in many areas.
I know, it sucks, but it is true. Few people want to give competitors a carte blanche to start with your work, and keep improvements to themselves.
I don't know much about Wasabit Systems or BSDi. But gosh, get your Apple facts right.
a) Apple's kernel is open source: having used GPL would not have made any difference here
b) The other BSDs have openly stated that Apple contributed back and thanked them for it.
Not only that, there are a lot of open source technologies that Apple has contributed to or is the main company behind it. In fact to find an example you didn't need to go any further than LLVM (discussed in this article). Apple is the company most strongly behind LLVM: they could continue in house without contributing any of the source, but they don't (and why should they? What advantage would it give them?).
Finally and I think the most important thing, is to ask: had those technologies been GPLed, would the companies have contributed back (as you seem to expect), dumped it all on their website, or simply have refused to use that code?
"Finally and I think the most important thing, is to ask: had those technologies been GPLed, [...], or simply have refused to use that code?"
Yes, who would use (L)GPL code in their systems?
$ gcc -v
gcc version 4.2.1 (Apple Inc. build 5664)
$ make -v
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
[...]
This program built for i386-apple-darwin10.0
$ troff -v
GNU troff (groff) version 1.19.2
It's true that LLVM and Webkit (based on KHTML, which was LGPL) are exceptions. However, it remains to be seen what they contribute back to LLVM in the future. Probably just those bits that will give them the tools edge.
LLVM is being developed by someone that Apple hired, why would they keep contributing back? Why wouldn't they have closed the doors on that back before the project got bigger? It is not in their best interest to do so.
The OpenBSM implementation in FreeBSD is based on the BSM code in Darwin.
Submitted by Apple to FreeBSD (apple.com email addresses)
Also, since all of this is BSD code, and those of us who developed under the BSD license agree to the fact that the code may be taken and used, and that works for us. Companies that know what is ultimately good for them will contribute back, maybe not all of their secret sauce so that they can continue to make money but at least part of it.
"Except for a necessary parts to make it useful. From someone close to the source (the last paragraph pretty much wraps it up)"
I am not sure what you are referring to. I'll try to reply to a few of the points raised in the article (but I have only skimmed it, so if I missed your point feel free to point it out).
Darwin is still being updated. Back in 2006 Apple didn't release the x86 code straight away, but I think they did so after a while.
DarwinBuild still exists and (apparently from their webpage) it is still being updated. This is a project supported by Apple whose only purpose is to build Darwin
As for Apple not including some of the drivers code in Darwin I can't say I am surprised. By looking at the fights between kernel people and drivers developers on Linux it seems a lot of companies (e.g NVidia) simply would not accept their secret sauce to be open sourced for fear it will reveal secrets of their architecture. Though correct me if I am wrong.
Finally the article you linked to asks why should people contribute to Darwin if Apple is less than helpful. Here I agree with you. Apple could do much more to help and encourage Darwin. But I don't see how this would be any different if the code were GPL.
And incidentally Android uses the same technique (keep development closed and open source the final version once it's being used) but everybody praises Android for being open source. (which is true, since they did open source _everything_ and Apple keeps things like Aqua closed source, but my point is that you don't hear quite the complains about Android's development model, as you do for Darwin's).
As for your second point, I think X-instance replied better than I could.
You prove my point: both companies, Apple with Darwin/OS X, and Google with Android are not really trying to develop a healthy ecosystem around these systems. But at the very least Google needs to provide the source to GPL'ed software (kernel modifications, etc.). If you have to publish changes anyway, at some point it becomes more attractive to work with upstream, since you cannot differentiate your product via that software. We still have to see if this happens, but there is a long of history of companies that did not contribute much initially, but are now working with kernel.org much more reliably. (E.g. many big-iron vendors.)
But from what I hear in the Linux world, having GPLed code didn't make all hardware producers write their own drivers and publish them. Some did, others released only simpler versions, and other didn't release anything and it was up to volunteer to do it.
Between this and a Darwin world where the latter group would release good drivers but keep them close sourced, I don't think there's any advantage for the community either way.
While I do agree that Apple is most definitely not trying to build a healthy ecosystem around Darwin (or in fact _any_ ecosystem at all...), Apple itself still publishes all of its Darwin code, and hence it would have complied with the licence had that been GPL. (the only exception being drivers but that's not up to Apple, as the situation in the Linux world showed).
Apple could have still kept Aqua and the rest closed source. So using Apple as an example of a company that took but didn't give back because of the licence seems wrong.
I just don't get why people always fight about licenses. I would think that developers have carefully thought which license they feel more comfortable with and that they know what they are getting in to. This sort of discussion won't really make anyone change their mind, they're jut a waste of time and energy for everyone involved.
Maybe it will be similar with LLVM. Back in the days Apple suggested to put a gcc front end to LLVM, but the GCC people were strongly against it. They seemed to have a really dislike of LLVM, or its licence, or the company behind it.
Eventually Apple (and the rest of the LLVM community) continued on their own. If GCC one day will become irrelevant, I think it's got only itself to blame, and a certain type of elitist arrogance. I wish (for them and the FOSS community in general) that the successes of LLVM will help them clean their house and keep going for decades and decades.