Hacker News new | past | comments | ask | show | jobs | submit login

This is my first reaction as well. How in the world do you end up with close to 10 different databases in production? And furthermore, most of which are totally proprietary? Just use Postgres...



Easy: These companies have a promotion process which require you to generate "impact" to get promoted. Using existing but boring technology which solves the problem in an efficient manner apparently doesn't show off how smart you are.

This is not a snarky comment from an outsider, I very much work at such a company. It's ridiculous.

Edit: But I must admit that most of this was probably developed 10 years ago when the ecosystem was much more limited. At least they now came to the conclusion that it's time to move on. I would not have recommended Postgres 10 years ago and even today there is no way it would work at the scale of Pinterest.


HBase is coming up on twenty years old.

In 2008, scaling was really hard. And the senior engineers at that point had been badly burned by trying to scale during the 2000-2005 period.

The comment about servers of the time being less powerful than smart phones of today is spot on.


So sad because so true.

Seeing working technology replaced because someone doesn't like it, get accolades for it, only to face the same limitations as old tech is just depressing.


You forgot the next step: The people who developed it move on to other companies and nobody wants to touch it.


Your application wants to use Postgres.

Kubernetes, which it runs on, wants etcd.

Keycloak wants to use Postgres and Redis for sessions.

Plausible Analytics wants to use Clickhouse and Postgres. Matomo specifically wants to use MySQL.

Lago Billing wants to use Postgres, Redis and Clickhouse.

Prometheus uses TSDB, but okay, it’s “just” files and a WAL. Wait a minute, so is Postgres…

Jaeger wants to use Cassandra or Elastic.

Thanos wants S3 APIs. So maybe you use Minio.

Okay. That’s 2024: in production, you will wind up with 7 different databases. At least.


>Just use Postgres

Now try running Postgres on vintage hardware from the period and you quickly see why we did what we did.


You would be surprised at how far a well-tuned and optimized Postgres instance will get you.

I've worked with so many teams who chase the newest and shiniest database, deploying hundreds of nodes, and spending hundreds of man-hours staffing a sysadmin team to maintain it, only to eventually replace it with an old fashioned Postgres box. You just need an experienced DBA.

Edit to add: many claims of "Postgres doesn't work at our scale" are frankly BS, and in large part due to an inexperienced ops team. Take a look at the top post on frontpage right now, for example: https://news.ycombinator.com/item?id=40372296


> You would be surprised at how far a well-tuned and optimized Postgres instance will get you.

I wouldn't be because I was hired after a team of very experienced DBAs saw their databases melt when trying to run analytics queries on the production database in 2012 for a telco.

We then moved to HBase so analysts could figure out what parts of the network needed extra towers installed for peak demand with 15 second intervals. You can do this today because the database sits on a machine that is between 10 to 100 times as capable as what we had back then and the number of people using cellphones hasn't increased substantially.

Again, don't look at this technology by what you can do today, but by what you could do back then.


Not sure if I can get you to 10, but different DBs work at different scales for different workloads. There's relational, key-value, caching, time series, data warehouse, and search index.


They are operating at a scale where you can’t “just use Postgres”




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: