I have dealt with qmail for ten years. It is a complete joy once you have it up and running. You literally can stop thinking about it because it always works. However, getting it up and running requires adding in antispam, antivirus, virtual mailboxes, some additional protective and administrative patches, some extra stuff for SMTP-AUTH, SSL/TLS, reporting, and perhaps a few more patches for fun.
Patches, you say? Well, I'm glad you really like source code, because that's exactly what you'll be dealing with to get all of this up and running. I've done it many times before manually and it takes around ten hours to do it.
While Aaron's point is that djb's code, vision and discipline are possibly unmatched (I concur on all counts), the software (principally qmail) is antiquated in the sense that nobody needs just that one part in order to run a semi-modern email system.
So, as you start downloading the .tgz for qmail and begin studying it like good campers, remember that this wonderful code is, by itself, not up to the job that it purports to serve.
-- Lots of love,
Someone who currently runs two qmail systems and will never go to the trouble of setting one up again
> I've done it many times before manually and it takes around ten hours to do it.
This is exactly why I've been using Postfix (or, preferably, get Google or someone else to handle it for me).
Easy to install is a feature, and a pretty close cousin of the "shipping" feature.
DJBs software is interesting mostly from an academic perspective. In the real world, it's hard to justify using something else, when apt-get gets the job done in literally seconds.
I haven't used qmail, but I use djbdns. The complete installation, possibly including a few patches, is less work than figuring out how to set up BIND's configuration. It's available in Debian from apt-get.
It's good that it's so easy to figure out how to use, because I touch it less than once a year, so I've completely forgotten about it. That's great software.
It is hard to imagine the serious, rational, objective argument that could exist for running BIND instead of djbdns, if those are your two choices. There are viable alternatives to qmail (Postfix and Exchange), but there aren't really viable alternatives for djbdns, and djbdns is less opinionated than qmail is.
I've only setup a single BIND server, and chose that due to previous exposure to the zone file format. It got the job done, pretty quickly even, but I'll definitely consider djbdns next time the need arises.
Next time around, take a look at PowerDNS. You like flat files as your zone backend? Supported. You want an SQL backend? Supported. You want an LDAP backend? Supported. And so on and so on.
DNSSEC is a disaster. Even if you liked the protocol today, having support for it in your DNS server wouldn't help you.
It's true though; thirteen root nameserver operators do run BIND. You know what's worse? The tens of millions of people that like The Black Eyed Peas. >shudder< - what's wrong with me?
If code I write is hard to deploy to your system, I don't think my code can be to blame. Rather, it's the fault of the system/language in which we are working.
If no one can use your code, it doesn't matter how beautiful, well-thought-out, or well-written it is. Software exists for the purpose of being executed. If your project cannot be deployed with a reasonable amount of effort, it's worse than bad software, it's a waste of time.
What about easy-to-install software that makes it easy for Bad People to ruin the Internet and to compromise your server (and potentially your entire organization)?
That is bad too. If I had to pick between "secure + customizable but hard to setup" and "easy to set up, but is a one giant gaping security problem", I think I would rather have the first.
Software is about collaboration, like a relationship; one person shares their ideas (qmail), another shares his (your site's configuration), and both parties do better than the sum of their parts. Easy-to-install software that doesn't require thought or communication is a one-way conversation; nice in the short term, but not something you want to be with for the rest of your life.
I hate this attitude. If your code targets my system, I should be able to install it without editing it.
You certainly should, but it isn't the coder's responsibility to make this possible. It's the language/system designer's.
Just because C doesn't let you deploy nicely has nothing to do with djb's ability to create beautifully architected code, which is what Aaron is praising. I'm sure djb could have written equally beautiful code in Squeak Smalltalk, and it would have been a breeze to deploy, but had no impact on how elegant the code itself was.
C does let you deploy nicely. A makefile and some discipline are almost all you need. Granted it's not always easy to write a sane makefile, but that is the coder's and not the installer's responsibility.
I don't think it's every C coder's responsibility to know how to make and ship good makefiles. It's certainly is helpful if she does, but it seems silly to say that's a responsibility, just as it isn't the responsibility of an artist to know the laws governing international sales of her artwork.
Sure, but it all depends on the coder's goals. If there's a competing option that's easier to install, prospective users may go with something else. If the coder doesn't care and isn't looking for users, then that's fine.
QMail doesn't target Linux? Oh, wait, you mean it doesn't target me and others who doen't want to invest the effort in installing it. That's fine, I use Postfix and it works well enough for my needs.
So what? Comparative programs have the same issue therefore it is not an issue? I'm not disputing that QMail is an improvement over Sendmail; in fact, I'm not disputing QMail at all. I've never deployed it, nor do I plan to do so.
I'm merely expressing my belief that in code that is meant to be used in production, facilitation of deployment is part of the problem scope. This is especially so when proper deployment is critical (as is the case with a mail server for example.) If QMail make deployment more difficult than it could be, then that is a problem with QMail. It might be the result of a worthwhile trade off. It might be better than it's competitors. But it's still a problem.
I agree with you. DJB wrote secure code that works very well and donated it to the public domain; if you want it to be Perfect For Your Needs, you are going to need to do that yourself. You are a unique snowflake, and your needs differ from DJB's.
"You suck for not doing enough work" is no way to treat someone giving you free software.
If DJB doesn't want to make his software useful, that's his right. We simply won't use it and the most perfect code in the world can pathetically fade into oblivion.
(read this message with a background of "arguing about the best programmer is anyway silly")
djb is in no way the best programmer of the world, but he is a very smart programmer, and probably an even better mathematician. And now, my arguments about why he is not the best programmer.
1) djb software is not the most elegant software around at all. Actually trying to write high performance and bug free software his style is the most procedural possible, with very little abstractions layers.
2) his software is well known to be uneasy to configure. There are qmail hackers around. When there are <name of a unix deamon>-hackers around it's a bad sign about the quality of the interface between the program and the rest of the system and the system administrator itself.
3) Djb played no role at all in the programming languages world. AFAIK he never suggested some new idea for programming languages or new directions that are now considered important. You can't be the best without being influential in the field.
4) Djb newer wrote big systems that were crucial for the programming world. Qmail and djbdns can disappear tomorrow and everything will be up and running anyway. GCC or the Linux Kernel are a different matter.
I can continue, but I think 1+2+3+4 are already enough to deny the argument aaronsw is trying to push in the article.
I question #3. Why is it admirable to play a role in the languages world? Shouldn't languages be a means to an end? If the end is good applications, DJB showed that you can produce good applications in C.
It seems like a lot of the energy around languages is frivolous. It's more fun to play with language features than to attack a real-world problem like writing a mail server.
As for #4, why does criticality matter? And why does the audience have to be programmers? Couldn't the author of a spreadsheet be as worthy as the author of a compiler?
Not that I'm endorsing DJB's nomination; I have pored through his source code for various reasons and have some reservations about it. But you could do much worse.
#3: sure, in general terms. Not true in my opinion for the best programmer of the world. Naturally he is "the one" that can master programming so well that it must be able to drive even programming language designers into more expressive / productive languages. djb can never be the one as in his code is very rare to find high level abstractions.
as for #4 it's very admirable to write real world software, but the best programmers in the world, like Joy or RMS tend to write big systems that can be used by the other hackers in the world, because to write a new operating system, a C compiler that live for decades contributing even to the development of new operating systems (It's hard to imagine Linux without GCC in some way), or world class text editors (vi, emacs) is not something everybody can do, but only the best programmers in the world.
Writing a text editor is a relatively easy task and it's more a matter of user interface than of implementation.
Besides, I don't see why writing a mailer daemon is less important than, say, a text editor.
To write a text editor today is very different that inventing the first VIsual editor that is a major step forward, or Emacs that was a similar impressive step forward at the time.
your points 2 and 3 directly conflict with each other. his software is "difficult" to configure because he does it in a non-standard way. individual files for options, installing everything in its own root directory, using /service for daemontools, etc.
he had these new ideas for software development, installation, and maintenance, so he implemented them and because they are different, people equate that with being difficult. he didn't like a lot of standard libraries, so a lot of his software uses his own routines (which has been extracted as libdjb - http://www.fefe.de/djb/) which is partially the reason for his impressive security record. i think you underestimate his impact on the unix software development community.
your fourth point is just wrong. a lot of ISPs rely on qmail and djbdns and to say that they wouldn't care if they just disappeared is just silly. sure there are alternatives, but so are there to linux and gcc.
I disagree, the buildings designed by Wright or Le Corbusier improved the field in many ways. They created a new architectonic system, it's like inventing a new programming language in some way.
this is a good point, but can be expressed with a blog post about "what are the qualities to be among the top programmers of the world?" instead of "djb is the best programmer of the world".
Still it is surely important to look at this great programmers in the aim to try to emulate all the stuff that they got right in order to improve ourself.
"As a UC Berkeley graduate student, Joy worked for Fabry's Computer Systems Research Group CSRG in managing the BSD support and rollout where many claim he was largely responsible for managing the authorship of BSD UNIX, from which sprang many modern forms of UNIX, including FreeBSD, NetBSD, and OpenBSD. Apple Inc. has based much of the Mac OS X kernel and OS Services on the BSD technology.
Some of his most notable contributions were the vi editor, NFS, and csh. Joy's prowess as a computer programmer is legendary, with an oft-told anecdote that he wrote the vi editor in a weekend. Joy denies this assertion.[2]"
Exactly my thought when I read this. IIRC he wrote one of the early TCP/IP implementations as well, which a lot of OS' still use a derivative of. And dns and mail rely upon.
The wikipedia article links specifically to a sadly abandoned project that Salon had years ago to document the free software revolution.
> What other field combines all these arts? Language, math, art, design, function. Programming is clearly in a class of its own. And, when it comes to programmers, who even competes with djb? Who else has worked to realize these amazing possibilities? Who else even knows they are there?
Yeah, only slightly obnoxious. Nothing worse than somebody so relatively new to a field that they still think it objectively is better/cooler/more fulfilling than every other field.
DJB's code is a lot of things, but when someone says "beautiful" is one of them, I start thinking about how I might quiz them to get them to prove that they've actually read it. Smarter coders than me have sat in rooms for nightlong studies of qmail and come to the conclusion that, while clever, the C code in qmail and djbdns has clearly been compiled down from some higher level language[1]. If that code is anything, it is idiosyncratic.
[1] Having asked this question directly to DJB in person, I can say that I am at least convinced he wrote this stuff in C.
Having read some of DJB's code as well, I also thought, "Huh?" when it was suggested that it was beautiful. One thing that would lead a person to believe the compilation argument is the almost complete lack of comments, as-short-as-possible variable names, etc.
The formatting, the heavy reliance on idiosyncratic CPP macros, and more than anything else the repetitively procedural nature of it is what set off alarm bells for us.
But really that's just the way he codes. It's very intricate. It's like a very ugly Swiss watch.
Having so clear a vision of how your architecture should behave that you human-compile it into idiosyncratic code that looks generated seems to be a very good definition of beautiful code, no? Maybe he hasn't had time to write a book about the "code patterns" which he's using, but that's hardly something to hold a grudge about.
Especially if you're a security person, I think it's uncontroversial to say that the design is beautiful. It's not only an elegant design, but it is the first major piece of systems code to take that design approach. To put it differently: most of major systems programs that have been proven secure in the last 10 years owe their design to qmail.
But that has nothing to do with the code, which is not only epsilon from assembly (Bernstein fully embraces the notion of writing code in high-level assembly), but also clever and concise almost to a fault.
As someone who ran qmail since it was originally released in beta, I also remember vividly Bernstein's original idea about configuration, which is that "configuring" your mail server with C code was more reasonable than learning another programming language (Sendmail "cf"). Which implies that a lot of the code in the interesting parts of qmail are less about design, and more about encoding mail routing policy as C code.
I don't specifically know of any overviews, but the main principles are fairly simple:
- Split the system into small components that do one thing and do them well.
- Give each part of the system the minimal set of privileges needed (if necessary by running as different users and set filesystem privileges accordingly). I.e. qmail has separate binaries for inbound smtp, pop3, managing the queue, local delivery, remote delivery and more.
- Make each part of the system communicate only via well defined interfaces (using pipes in qmail) where it is explicitly assumed that you can't really trust the sender.
- Don't ever use library functions that don't length check things. Then again he uses his own stdio replacement, and his own string functions.
qmail has a good design, but most of this is implementation of an existing well known concept, Principle of Least Privilege. Similar apps like Postfix owe more to whoever came up with that theory than they do to DJB IMHO.
If you write code that looks like it was generated with e.g. yacc, then you probably would have been better off writing it in yacc and bundling that as the source. There's nothing glamorous in being a human compiler. Other people will have to touch the code at some point.
I haven't read the qmail source (just some design docs about how the whole system fits together, which I found rather impressive), so I'm not talking specifically about that.
NB: Qmail's license wasn't public domain until 2007. (Also, damn, tptacek, you're fast! I deleted that part, since I'm talking about auto-generated code in general.)
(I'm in 100% full-on maximum overdrive procrastination mode today, since what I need to get done is to script and record a screencast of my app, and I'm frozen up about where to start with it. Sorry for being so fast to respond).
While this is certainly impressive, the number of bugs is not a good metric for measuring the effectiveness of a programmer. A programmer is only as good as the business problems she solves.
One bug in a life-critical application could be catastrophic. On the other hand, a programmer that makes sure their code is completely bug-free would not do well in a domain where time-to-market is critical and quality of service is not.
I agree, but aaronsw must come at an issue as a raging partisan. I know that in economics, philosophy, politics, and apparently best programmer ever!!!, he always comes at the issue with a deeply contentious point of view from an entrenched idealogical camp.
It's kind of a shame because of the the advantages of being young is being able to approach a field without having to take sides. Meaningful dialog and inquiry void of motivated reasoning are beautiful things. Unfortunately, some people need an argument.
the advantages of being young is being able to approach a field without having to take sides.
You must be describing your youth, not mine! My experience with youth is that it was the period of my life in which the whites were blindingly white, the blacks impossibly back, and there were no colours or shades of grey to distract us from our belief that we knew exactly what was right and what was wrong.
And thank goodness, because a belief in right and wrong can propel you to change the world while you have the energy to carry it through :-)
I have to agree with you on JQuery. It runs on multiple browsers too and performs exceptionally well. It's gotta be one of my all time favorite pieces of software.
It's hard to overstate how great jQuery is. I'm always surprised at how easy it makes things. I remember the bad old days, spending hours researching the different ways to reference dom objects, trying to find a method that worked in more than one browser. With jQuery, you really can just code it once, and it works everywhere.
Only two others would equal DJB in terms of having more than one major well known app (a mail server and a DNS server). In my experience, I wouldn't say djbdns is particularly widely deployed...
Linus Torvalds
* The Linux Kernel
* git
Fabrice Bellard
* FFMpeg (if you've done anything with video in the last 10 years, you've used this)
* QEmu (basis for QEmu emulator and userspace portion of KVM VM). Ubuntu's open EC2-API compatible elastic cloud app uses this, as does Red Hat Advanced Platform.
If there are any others, let me know. Mongrel's pretty popular, if Lamson does the same maybe we could add Zed.
What is this, the Hill Valley International Programming Tournament?
People have arguments like this all the time in the sports sector, and at least there you can come up with all kinds of statistics and recordings, but what do you have for programmers? If they produced Open Source, you might have the end result of their work, but that would exclude a large part of the work force. And even then, what are your points of measurements? Just the amount of bugs? Total? Per year? Per line? Meh.
And as a final, somewhat unrelated note: Which I have nothing against Mr. Bernstein himself, I've got a pretty low opinion of most djb fanboys I've encountered in the past...
This is slightly off-topic as it's not about djb, but about qmail. Still:
I have some experience with qmail. While I agree that it's generally very nice and certainly incredibly bug free, that quality comes at a price: Lack of features.
The moment you start adding stuff: Server-side filtering (like sieve), LMTP, virus checking, spam filtering, virtual users - all that stuff you need to run a modern mail server - you will have to patch qmail.
And by manually patching your installation, you are picking up responsibility: Now that you are not running stock qmail, but something you patched and installed yourself, you will no longer be able to rely on your distribution for security fixes and you'll have to keep updated and repatch yourself.
And while qmail is beautiful and bugfree, the same can not always be said for the additional patches, so you WILL be patching.
If you get all the patches for the functionality you need to actually play together that is.
If qmail solves your issue and you are prepared for this, then go with qmail. If you need a ton of functionality built-into your mailserver, probably go Exim and if you need good compromise between featureset, architecture and security, then you'll probably take postfix.
Please avoid Exim. Even if you can't stomach qmail, which is an acquired taste, you don't have to compromise on security. Smart people can disagree about whether Postfix is _as_ secure as qmail, but none of them disagree that it is at least _very_ secure.
I have a feeling that a large portion of the best programmers work in absolute obscurity.
I worked for a long time with guys that wrote embedded software. Absolutely brilliant guys that never get their due because no one knows how they were.
> No other programmer has this kind of track record. Donald Knuth probably comes closest, but his diary about writing TeX (printed in Literate Programming) shows how he kept finding bugs for years and never expected to be finished, only to get closer and closer (thus the odd version numbering scheme).
I don't want to be dismissive but I think TeX is more complex and uses non trivial algorithms. So it's like comparing an apple to an orange.
As someone who has hacked mail servers, written DNS servers, and come within 100 miles of typesetting, I'm going to throw in my vote that TeX addresses a much harder problem than qmail.
No contest there, but qmail is far from 'trivial'.
If you're going to compare qmail to something else though I think it should be compared to sendmail (or postfix for that matter), not to TeX.
Personally I think djb is a great coder, but there are quite a few of those around.
Programming is not a single-valued enterprise anyway, so best is a very hard to measure quantity, as good as meaningless. But I know 'bad' when I see it :)
The algorithms and pure theoretical CS in qmail actually is trivial. From what I remember, pretty much the only genuinely interesting thing in it is how he implemented his hash table.
From a systems programming perspective, qmail is not only nontrivial, but actually groundbreaking. His allocator design, the way he architected his libc replacement, the extent to which he takes advantage of bare-metal Unix programming (look at his queue notification mechanism), it's all really cool stuff.
But there's a big difference between theoretical CS and systems programming, and the parent commenter is right to point out that TeX is more complicated from a CS perspective than qmail is.
Despite the purple prose, I generally agree with Aaron's sentiments about great programming ("the best", I can't say, but definitely "great"). He is, however, also very rude, at least in his online persona. Having "a forceful, uncompromising vision" is an orthogonal issue to how you choose to communicate.
daemontools and tinydns are far, far better examples of brilliance than qmail, in my opinion. Especially tinydns comes far closer to meeting modern needs than qmail, for its domain. I'll never go back to the horror of bind config files.
djb wrote the code that won the EngineYard SHA1 challenge. His code was processing 800 million sha1 hashes per second. He wrote CUDA and C implementations. His sha1 implementation was 12 times faster than OpenSSL.
This article is pathologically sick. Couldn't read it any more, to painful.
qmail gives you pain, too. It is completly useless without applying patches, which are mostly of poor quality. And it misleads you to do most unperformant things around it. Like writing shellscripts which fork() like hell for every mail there is to process.
Yes, that's true for most domains. All the other servers have to cache responses from those two servers. (For the record, the two yp.to servers are currently djb's machines dancer and forcewall.)
What would happen if he suddenly decided to unplug dancer and forcewall forever, would all the other DNS servers in the world fail to resolve yp.to after their cached responses expire?
I love djb and his tools but, as someone who hacked djbdns extensively for a research project, I think the project misses empathy for other programmers. It's hard to follow, poorly documented, and too clever sometimes. That said, if we judge code only on performance, then yeah it's rock solid.
There are several less known project with really passionate coding, especially http://nginx.net/.
postfix, sqlite and many other projects could be compared with qmail.
I don't know whether it's legal, but http://www.interact-sw.co.uk/altair/index2.html has an annotated disassembly of Altair BASIC. I haven't given it more than the briefest glance myself, but allegedly there's some very nice stuff there. (Of course it's from a different era and no one outside deeply-embedded-land writes that sort of code any more other than for a twisted kind of fun.)
He wrote Microsoft's original Basic, probably parts of other languages. DOS must have been changed a lot after Microsoft bought it. He may have written some of the new stuff.
Patches, you say? Well, I'm glad you really like source code, because that's exactly what you'll be dealing with to get all of this up and running. I've done it many times before manually and it takes around ten hours to do it.
While Aaron's point is that djb's code, vision and discipline are possibly unmatched (I concur on all counts), the software (principally qmail) is antiquated in the sense that nobody needs just that one part in order to run a semi-modern email system.
So, as you start downloading the .tgz for qmail and begin studying it like good campers, remember that this wonderful code is, by itself, not up to the job that it purports to serve.
-- Lots of love,
Someone who currently runs two qmail systems and will never go to the trouble of setting one up again