This is 100% correct. We're in the process of doing this now with our products (and our internal systems, also), and it works well.
When you have products and systems that have been around for a decade or more, major portions of them become outdated and need to be rewritten (or largely rewritten, the two are synonymous to me). Best practices in crypto change, operating systems and hardware improve/change, etc., and if you want to keep generating value, you had better change along with it. You can see where organizations don't do this: the developers force weird constraints on IT like needing to use old, obsolete versions of operating systems because the software won't run on newer versions.
Along these lines, one of the problems that I think exists with the software industry today is an inability among developers to recognize that software sticks around a lot longer than one might originally envision. And this phenomenon only gets worse (better, for the end user) as the value provided by the software increases. It's a bit of a Faustian bargain: everyone wants their software to be used and provide value, but often don't realize the "soft commitments" being made in the background that can tie you (or the business) to the code for years (or decades).
To me, the distinction is between a society where individuals are free to choose to rent or own, based upon their private circumstances, vs. a society where individuals are forced to rent because it is impossible to own. It seems to me like we're heading towards the latter, and that's not good.
Key-value stores are fine if you are always simply grabbing a single row/tuple from a single dataset. Any time you move beyond that, you're in for a world of hurt compared to a relational database that uses SQL.
The reasons are:
1) You often need a transactional, consistent view of the data across multiple datasets (tables). Relational databases using SQL provide that, key-value stores may or may not (most likely not).
2) Any time you're doing any kind of aggregation, key-value stores will require that you pull all data in locally and perform the operation there. It's questionable whether the local environment even has the resources to accomplish such a task, and the efficiency of such operations will be dismal compared to a SQL-based relational database server.
Key-value stores are, from an interface perspective, very much like the older ISAM databases that were popular before SQL and relational databases gained a foothold in the industry. They also recycled the exact same problems that SQL-based relational databases solved so elegantly (round and round we go).
Another way to put it is this: leaving aside scaling for now, SQL-based relational databases can typically do everything that a key-value store can do, whereas key-value stores cannot typically do everything that a SQL-based relational database can do.
I wasn't so much asking about relational databases vs key/value stores, but rather why should I implement things like advanced sorting, mathematical formulas, etc in SQL rather than in the application code that fetches data from SQL?
It's simply more efficient? If I have a table with millions of rows, I'm not going to bring down the entire data set on every request just to sort by latest and grab the 10 most recently updated rows that haven't been soft deleted (WHERE active=1 ORDER BY updated_at DESC LIMIT 10).
- Advanced sorting plus pagination comes to mind. How would you do that without pulling unnecessary rows?
- GROUP_CONCAT or STUFF in SQL Server already decreases the number of rows. His example has one row returned instead of 3.
- Aggregating in SQL would also reduce number of rows returned instead of pulling all the aggregated rows to code and doing it there
- I would guess RDBMS implementations are in something faster than the ORM is implemented in and their code is more optimized - that would be speculation.
Also, his examples are like the tip of the iceberg. There are all kinds of things like windowing functions and common table expressions that have so much power to get exactly as much data as you need right at the RDBMS without having to carry that baggage anywhere else.
Anything that deals with the source data, such as aggregation, sorting, or partitioning, should be done with SQL.
Anything that deals with the formatting/display of the columns in the result set rows can be done in SQL, but doesn't have to be done in SQL. It's really a matter of convenience to do it in SQL, especially when dealing with multiple front-end languages, because it means that the formatting is done in one place.
Our product, Elevate Web Builder, has been using component-based visual controls with no CSS since 2014. It's modeled on the traditional way of doing components/controls in Delphi's VCL and .NET's WinForms, with a splash of WPF in terms of how control interfaces work.
I tested this recently (Delphi XE6), and it definitely is as-described: you get straight jump instructions with enough case statement branches, and it is very fast.
Barry Kelly worked on the Delphi compiler, and I believe he comments here on Hacker News occasionally.
The interesting part to me was the variations in the emitted instructions, based upon the source. IOW, it would be easy to mistakenly think that you weren't going to get jumps if you only used a few case branches to test things out.
I know that you're probably talking a lot older than one century, but as a minor counterpoint:
Our house (in Buffalo, NY area) turned 100 this year, and there are thousands of houses just like it in the Buffalo area. In fact, most of the housing that was built in the late 1800s and early 1900s is still standing:
> In fact, most of the housing that was built in the late 1800s and early 1900s is still standing.
I don't think that the linked article actually says that.
What it does say is that Buffalo has the oldest housing stock in the country with a median construction year of 1954. I would guess that to be mostly an artifact of the declining population of the area, which peaked between 1960 and 1970.
Yeah, I kind of conflated my anecdote with the reference in support of my anecdote. I'll have to see if I can dig up more data on this.
As for population, you have to be careful with the statistics on this because, while the area did lose population since the 70s, there was also a lot of flight from the city to the suburbs over the same time period. So, it is often more instructive to look at the total population of western NY instead of just the city of Buffalo.
the issue with old housing in the end is simply heat isolation. Old houses leak away alot of heat (especially if deep cracks are present in the brick and mortar, which is very expensive to fix). Which in term will mean you pay a significant amount of extra cash to keep the heat on. Renovation is possible ofcourse, but not always worth it?
Many of the older homes here are wood (typically, balloon frame) with a sandstone foundation, so you can reduce the amount of heat loss by making sure that the sandstone mortar is in good shape, making sure that all windows and entrances are properly-sealed, and strategic use of insulation. I say "strategic" here because you have to be careful about making the house too tight, which can cause moisture to get trapped and result in very bad things like sill rot.
The key is that these old homes have very good "bones", so even a gut job can use the original structure intact. Combine that with the fact that Buffalo's real estate is relatively inexpensive still (it's starting to get up there, though), and extensive renovations aren't too bad on the wallet.
I need to ask you an honest question: have you every run your own business ?
If not, you should try it, it's a really eye-opening experience that will give you really valuable insight into the world that we live in. The one thing that you will learn is that we are all creatures of the environment we operate in, and very few of us actually have economic/political power that extends very far, so one should always be very careful about ascribing ulterior motives to other people. Most of the time, people just want to make a decent amount of money so that they can have a nice life with their families.
We do have ways of self-publishing or paying creators directly: the internet. Anyone can pop up a web site and start selling their wares with minimal fuss. It costs a couple of thousand bucks to set up an internet-based company in the US, with the majority of the money going to incorporation. Payment processing services, payroll services, tax/accounting services, hosting services, etc. are all super-cheap and don't require a lot of work.
Sometimes I have to stop myself from constantly grumbling "kids these days...", but wow, I'm seeing a lot of young people erecting mental barriers to acting on their dreams and aspirations. It's never been easier to make money in this country, but you have to a) put in the hours, b) stop giving away your stuff for free, and c) manage your expectations - nobody is saying that you're going to win the lottery by becoming the next Google or FB.
Yes, I would like someone here to provide an answer to this question.
For example: We're just wrapping up a new release of one of our products that I've been working on since August of 2017. It has over 1400 hours of work dedicated to it (single developer - myself). There's no way that we could possibly dedicate that amount of time to a software project without knowing that we could recoup the costs.
The trick is almost all creatives are already paid in the model most fitting the production of information - people that want the fruits of their labor pay them to make it. Very few software engineers, for example, are living off copyrighted sold copies of the software they wrote. The company they work for might, or some other knob in the chain, but for 99% of us we are not directly using copyright as our source of income - someone else is.
And that is the trick. People want software. We threw this extra, pointless step in the middle between "people C want X software" and "pay people D to make X" in the form of "have publishers / corporations P pay people D to make X to sell to C".
Every conservation on this always functionally boils down to "but that is so different from the way thing are now!" which of course it is. Its the same reason why the market for prohibited drugs is so much more convoluted, dangerous, and disconnected from regular commerce - you make things more complicated, like criminalizing posesssion of the good, or enabling the profiteering off information, and the market will warp to both meet demand and maximize profit using the framework the government provides to do so.
Its really no different from how you are told to get started in music or art or even how to get a job programming. You make stuff for free to start out to get a portfolio of work proving you know what you are doing. Once you can prove your ability you ask for money to make more, be it words, art, designs, code... its how the world has worked for all time. It is only in the last two hundred years this model of "make stuff and ask for money later with the backing power of the state" perverted commerce. And all of us commoners never even got to benefit. We still work in classical employment. We still need to do the work for free to prove we can make more information in the future. We still get paid for our time, not the fruits of our labor. Which is fine - its how the whole rest of the economy works!
There is only one scarce thing in the creation of new ideas and that is your time. You should be charging for it. Its all thats worth anything in the equation of creativity. Most people do charge for it with salaries. The world still predominantly runs in tandem with reality. Its only a very few people who get to throw a state backed wrench into economics to rent seek already made ideas.
So, how is getting paid for one's time any different from being paid for the value of one's labor ? Do you think that economic theory doesn't apply in that case, and that, somehow, a person isn't going to just simply demand a huge salary that results in the same compensation ?
And how to you prevent someone from just creating a piece of software, keeping it to themselves, and then charging others if they want a private copy of it ? Are you going to demand that software developers release their code to the public, and enforce that through the state ?
What about live performances ? Are you going to mandate that performers invoice each person in attendance for their time ?
When you have products and systems that have been around for a decade or more, major portions of them become outdated and need to be rewritten (or largely rewritten, the two are synonymous to me). Best practices in crypto change, operating systems and hardware improve/change, etc., and if you want to keep generating value, you had better change along with it. You can see where organizations don't do this: the developers force weird constraints on IT like needing to use old, obsolete versions of operating systems because the software won't run on newer versions.
Along these lines, one of the problems that I think exists with the software industry today is an inability among developers to recognize that software sticks around a lot longer than one might originally envision. And this phenomenon only gets worse (better, for the end user) as the value provided by the software increases. It's a bit of a Faustian bargain: everyone wants their software to be used and provide value, but often don't realize the "soft commitments" being made in the background that can tie you (or the business) to the code for years (or decades).