There are plenty of people who build here on HN (more than most other sites) and the requirements are pretty clearly described in the article.
While it's not as simple as a Go program on a VPS, there is certainly a lot of unnecessary overhead here. I think you underestimate just how much poor and wasteful engineering there is out there.
> While it's not as simple as a Go program on a VPS, there is certainly a lot of unnecessary overhead here. I think you underestimate just how much poor and wasteful engineering there is out there.
I don't under estimate poor and wasteful engineering at all, but that's not what I saw in the article.
Serving traffic is a single element of their design. They also designed for security, redundancy, and observability. All with their own solutions because using a service or a cloud provider would be too costly. With that in mind, it's not a charitable view to think they didn't explore low hanging fruits like "make the server in Go". If you think you can do better, detail in depth how and solve all of their requirements vs. the single piece you're familiar with.
And if you can do the above holistically, for an order of magnitude below their costs, it sounds like I need to get in touch to throw money at you.
My background is in adtech, which is a unique mix of massive scale, dynamic logic, strict latency requirements, and geographical distribution. I've built complete ad platforms by myself for 3 different companies now so I can confidently say that this is not a difficult scenario. It's a ready-heavy content site with very little interactivity or complexity to each page and can be made much simpler, faster and cheaper.
> "detail in depth how "
This thing seems to be little more than a very complex API and SPA sitting on top of Elasticsearch. These frontend/backend sites are almost always a poor choice compared to a simple server-side framework that just generates pages. ES itself is probably unnecessary depending on the requirements of their search (it doesn't seem to be actual full text indexing of the content but just the metadata). The security and observability also tends to be a problem of their own making and a symptom of too much complexity.
> My background is in adtech, which is a unique mix of massive scale, dynamic logic, strict latency requirements, and geographical distribution. I've built complete ad platforms by myself for 3 different companies now so I can confidently say that this is not a difficult scenario. It's a ready-heavy content site with very little interactivity or complexity to each page and can be made much simpler, faster and cheaper.
I don't dispute this or your credentials. You've built critical systems in a space where it was a core of the business. If given time, and resources, I have no doubt you could build a custom solution to their problem that was more efficient.
Unstated in this is the type of business MangaDex is, which I have the following assumptions about. I don't think it's unfair to assume that we're mostly on the same page here:
- Small to mid size, at most
- Small engineering team. Need to develop, deploy, support, and maintain solutions.
- Lacks deep systems expertise, or is unable to attract talent that has that expertise ($)
These characteristics are very common in our space. To solve their technical problems, most of the time, they reach for an open source solution (after examining the alternatives like a service).
Now the question is given those constraints, and their other business requirements, how do they best optimize for dimensions they care about? Everything is a trade-off. Everyone who builds knows this. It's unkind to pretend this is a purely technical exercise. And after reading their article, it's obvious they know some of trade-offs they're making, so it's unkind to suggest a naive solution that does nothing but make you feel smarter. I'm not saying you did the above, but some of these comments are outrageous.
While it's not as simple as a Go program on a VPS, there is certainly a lot of unnecessary overhead here. I think you underestimate just how much poor and wasteful engineering there is out there.