I think a lot of it has to do with the fact that software engineering as a discipline is so huge that often times there are giant swaths of communities struggling with concepts and how to solve problems that really smart people already figured out the solutions to 25 years ago. Like how on the Unicorn! blog post Github wrote a while ago, someone was amazed[1] that they hadn't heard of known decades-old things like the pre-forking socket server.
It would be an interesting experiment to get a bunch of luminaries from different computing disciplines in a room together and to have them all talk about problems they have and see if anyone has any solutions.
I think what you're seeing here is endemic of teams of almost entirely junior engineers with insufficient experience (especially in startups), rather than the software engineering space being inordinately large.
Regarding GitHub's seeming rediscovery of the pre-fork model, I too was pretty surprised -- largely because the model was long-since dated by the time they opted for it, as it doesn't scale well on modern multi-core/multi-cpu hardware. This original HN thread on Unicorn had a vibrant discussion on the subject -- the differences of opinion seemed to boil down to experience and subjective biases: http://news.ycombinator.com/item?id=872361
> It would be an interesting experiment to get a bunch of luminaries from different computing disciplines in a room together and to have them all talk about problems they have and see if anyone has any solutions.
I think it would get considerably more interesting if you brought in a bunch of people from other disciplines as well. Exposure to alternate disciplines and the models they create for the world is, I think, one of the most valuable things one can do to enhance their understanding of their preferred field of study.
I sort of assume the entire field of Bioinformatics developed out of a couple of colleagues from Biology and CS having a beer and a similar situation unfolding.
It would be an interesting experiment to get a bunch of luminaries from different computing disciplines in a room together and to have them all talk about problems they have and see if anyone has any solutions.
[1] https://github.com/blog/517-unicorn#comment-3124