I work in a huge company. Being consumed with what might be tomorrow is taken to an extreme. Given what I hear from other companies, this is very common. Sad part is, it never works. Designs don't ever match up with real world requirements (they match up very well to stated requirements). Anticipated changes never happen, but other changes do, and ... then of course a rewrite is needed, after attempting to get 2-3 unanticipated changes built into the "agile" design. "Technical debt", is the explanation, often without any further clarification.
We have inventory control systems that have their own database server (including disk format, their own indexing system, ...), own query language (because, ... euhm ... needless to say, someone wrote a script that dumps their own database into a mysql, and deletes and rebuilds the mysql every 2 hours, and everyone uses that to query the data. These days, even including the team that built the product). Impressive code (or I should say, impressive that it doesn't burn down our data more often). The point of doing/having done this ? It makes everyone's life more complicated, which seems to mean more reason to promote the participants.
Of course, failure of designs and over complicated designs is always seen as a reason to do more design (every postmortem, every time), with more anticipated future evolution, instead of less. Better and more accurate product managers, better and more accurate requirements (that are never going to come). I shudder to think what happens after rewriting databases from scratch has become the norm. More product managers to collect better requirements from people who don't know their own requirements. And of course, none of them ever see any need whatsoever to understand the problem they're solving.
The scary part is, rewriting database engines starting from a basic key-value store ("nosql") does in fact seem to be the norm now, even on hacker news. App-specific query languages are everywhere. People are I'm not kidding proud of doing this. Nobody seems to realize that this just isn't worth it.
Cynicals say that this is simply "CV-first engineering". Of course the database system for your company ticketing system needs to scale to 50000 employees, and 1 billion customers, never mind that you have 10 at the moment. When you go and interview at Google/Facebook/Amazon, and they ask you "what have you built", are you going to answer "a system that works really well for our 100-person 500-customers company" which runs really well on a single cloud server. They'll laugh you out of the room. (of course reality is that no, they won't laugh you out of the room)
A very interesting answer. I could very much identify with what your saying.
> Designs don't ever match up with real world requirements (they match up very well to stated requirements).
Perhaps the most important discipline in computing isn't anything related to technology, but is instead "requirements analysis". What does your company generally use to communicate and record requirements - use cases/stories/other? Are we just still struggling with the same old problem - data can be modeled easily, but not complex behavior?
> More product managers to collect better requirements from people who don't know their own requirements. And of course, none of them ever see any need whatsoever to understand the problem they're solving.
Do you think these people don't know their own requirements because: aspects of the complex underlying real-world problem or system are difficult or unknown to them; they are trying to describe something new in their imagination; the mechanism for writing the requirement is vague meaning it inevitably needs to be further decomposed; something else?
> The scary part is, rewriting database engines starting from a basic key-value store ("nosql") does in fact seem to be the norm now, even on hacker news. App-specific query languages are everywhere. People are I'm not kidding proud of doing this. Nobody seems to realize that this just isn't worth it.
How do proponents of these "custom" database engines justify them if they are not really needed? Is management clueless?
We have inventory control systems that have their own database server (including disk format, their own indexing system, ...), own query language (because, ... euhm ... needless to say, someone wrote a script that dumps their own database into a mysql, and deletes and rebuilds the mysql every 2 hours, and everyone uses that to query the data. These days, even including the team that built the product). Impressive code (or I should say, impressive that it doesn't burn down our data more often). The point of doing/having done this ? It makes everyone's life more complicated, which seems to mean more reason to promote the participants.
Of course, failure of designs and over complicated designs is always seen as a reason to do more design (every postmortem, every time), with more anticipated future evolution, instead of less. Better and more accurate product managers, better and more accurate requirements (that are never going to come). I shudder to think what happens after rewriting databases from scratch has become the norm. More product managers to collect better requirements from people who don't know their own requirements. And of course, none of them ever see any need whatsoever to understand the problem they're solving.
The scary part is, rewriting database engines starting from a basic key-value store ("nosql") does in fact seem to be the norm now, even on hacker news. App-specific query languages are everywhere. People are I'm not kidding proud of doing this. Nobody seems to realize that this just isn't worth it.
Cynicals say that this is simply "CV-first engineering". Of course the database system for your company ticketing system needs to scale to 50000 employees, and 1 billion customers, never mind that you have 10 at the moment. When you go and interview at Google/Facebook/Amazon, and they ask you "what have you built", are you going to answer "a system that works really well for our 100-person 500-customers company" which runs really well on a single cloud server. They'll laugh you out of the room. (of course reality is that no, they won't laugh you out of the room)