I understand your excitement for being able to handle a decent amount of requests on such a small server, but just like many other websites that get on the frontpage of HN, your site is taking multiple seconds to load for me, depending on when I refresh.
As you said in your post, adding caching to your site increased your throughput by ~20% (or +10/req/sec). What you and other sites seem to lack is a more distributed caching, a la CloudFlare, S3 CloudFront, Azure CDN, etc. Those last two only really work well for a static site, however as mentioned in your post that's essentially what you're serving.
While I'm all for having a free-as-in-freedom hosting solution and keeping things lean, the internet is a fickle beast, and nothing looks worse for a company who posts on HN when their technology-oriented site can't handle a few thousand requests per minute. (Or in this case, when a blog claims to handle 4.2M requests a day -- 2.9k req/min)
Looks fine over here, and he doesn't have to route through a fucking Internet gatekeeper like Cloudflare or Amazon... let's enjoy this golden era before Chrome starts flagging any site which isn't fronted by a "reputable" cache like Cloudflare, Amazon, or whatever Google decides to introduce.
It would also be possible for OP to spin up their own Redis cache, and have multiple POPs near their target audience, and handle DoS type attacks against their site if need be, and easily be able to brush aside bot traffic, and...
Not all the above apply to a hobby-blog style site, but I wasn't referring only to OP's site in my original comment.
I understand that not everyone needs to feed into "fucking Internet gatekeeper"s as you described, but the fact that they provide valuable services is undeniable. They make a complex operation -- one that could mean the difference between a company being able to sell their product or not -- simple.
For sure, OP's site is handling this much better than most. And like I said, it's not every time that it takes multiple seconds. Some websites featured on HN/Reddit don't load at all when under load. However I was able to get it to take ~30s to load multiple times, over a period of around 10 minutes.
The important message there is that if you can change your problem from serving slow dynamic content to serving static content you can gain enormous performance benefits.
Whether that means actually using static sites for stuff that can be static or just properly caching expensive things. Even dynamic content doesn't have to be slow, but many CMS are seriously inefficient without a cache. I'm not really blaming the CMSes entirely here, part of that is because they need to be extremely flexible, but once you need dozens of DB queries per page it'll fall over quickly on small hardware.
We're using Next.js at my current company with a custom MongoDB based CMS.
Next has a thing called Incremental Static Regeneration[0] which allows us to grab the top ~100 pages from the CMS at build time, generate the pages, then cache them for however long we want. The rest of the pages are grabbed when requested, then are cached for the same amount of time. After the time, they're re-grabbed from the DB, then re-cached. Overall I think we're down to around 5-10% of the way things were done before, which was -- you guessed it -- hit the DB on every page load _just in case_.
Sit the Next.js site behind CloudFlare, and then we also don't really pay data transfer costs. Our servers are just low-tier GKE nodes, and we run around 3k/visitors at any given time, sometimes spiking up to 8k concurrent.
Even database queries aren't that slow on reasonable hardware, as long as the queries are simple. The problem appears once you have dozens of DB queries per page. It's really not a fair comparison to the site this topic is about, but for trivial queries you can easily get a few thousand requests per second out of Postgres on desktop hardware without any real tuning as long as the DB fits into memory.
But static content is of course still much faster and also much simpler.
Are you sure that's related to the server itself? The page loads instantly for me and the DNS is still resolving to an OVH IP address.
Timing info from Firefox:
Blocked: 0ms
DNS resolution: 8ms
Connecting: 9ms
TLS setup: 12ms
Sending: 0ms
Waiting: 30ms
The very last resource (favicon.ico) loaded after 466ms and that's mostly because of the other files being requested only after the CSS has come in (after about 195ms). All in all the entire site (without the Matomo tracking JS) loaded in half a second.
Maybe the website has switched hosts in the last ten minutes, I guess, but I doubt it. I think this is more likely to be a problem related to distance to origin and saturation of the underlying connection.
Working in the space, that's one of the more frustrating things to see on HN/Reddit/etc. It's not a complex or niche thing, and especially for sites that only make profit when people can actually visit them, it's kind of a necessity to stay up as much as possible.
I don't have a "Do all These for a Fast Website" list handy, but here are some key points I've found can be applied to most sites:
- Make sites that are fast by default: Small bytes sent over the wire, beyond just initial page load, too. Yes, that does mean that your giant Google Tag Manager/Analytics/3rd party script is bloated. Reach out to 3rd parties about reducing their payload size, it's saved me several MB over the years. Also, not writing efficient CSS is a huge killer when it comes to byte site. Devs shouldn't "leave it just in case" when it comes to code, you have version control for a reason. And when a new feature comes out, clear out the old cruft.
- Avoid unnecessary DB calls: Obviously you need to get the data onto the page somehow, but if you can server-side render, then cache that result, you're reducing the overall calls to the DB. Also, optimizing queries to return only-what-you-need responses helps reducing total bytes over the wire
- Balance between Server and Client side: Not only are servers getting more powerful, so are client devices. Some logic can be offloaded to clients in most cases, but there needs to be a balance. Business-critical logic should probably be done server side, but things like pagination & sorting -- so long as they client will likely see or use all the data -- is fine in my book. Having 2000 rows of JSON in memory is totally OK, but rendering 2000 at once might cause some issues. Again, balance
- Hopping on the latest-and-greatest bandwagon isn't the best: Devs hate re-writing the site every 6 months, and really the newest framework might not be the best for your use case. Keep up to date with new technology, but saying "not for me" is fine.
- Don't let (non technical) managers make technology decision: See above. More often than not, C-levels want to use shiny new things they read an article about on LinkedIn once, no matter if it fits the needs of the company or not. Thankfully I've only been at one place that was like that, but while I was there it was hell. Current VP was an original developer on the site back in the early 00's, so he knows how to deflect BS for us. That VP also knows that he's outdated in his knowledge by now, so he trust the Devs to make technical decisions that are best for the company.
Working extremely fast for me right now. Your post was about 20 minutes ago. I don't know how HN traffic fluctuates, but it seems really solid compared to most sites.
> Parts of the blog posts are cached using memcached for 10 mins
That means Django needs to accept the request, route it, pull the data from memcached, render the template.
For such a site I'd just set the `Cache-Control` headers and stick Varnish in-front of it acting as a reverse proxy. That'd likely increase the page load times significantly and make the backend simpler not worrying about manually caching in memcached and just setting the correct `Cache-Control` http header.
As it's budget hosting i'd probably not even bother with Varnish and outsource that to Cloudflares generous free tier, it's cheating as your server (Origin) isn't doing 4.2m requests but the practicality is really convenient.
Hey OP, in case it wasn't clear from my original comment, I am impressed with how stable your site is! "A few seconds" is wonderful for a #1 post, and far better than what usually happens to lightly-hosted sites that get to this point. Those are usually timeouts or outright failures to connect.
I just figured I'd start some conversation on the post, since there wasn't any comments when I initially looked. For better or for worse it seems like I got people talking.
As you said in your post, adding caching to your site increased your throughput by ~20% (or +10/req/sec). What you and other sites seem to lack is a more distributed caching, a la CloudFlare, S3 CloudFront, Azure CDN, etc. Those last two only really work well for a static site, however as mentioned in your post that's essentially what you're serving.
While I'm all for having a free-as-in-freedom hosting solution and keeping things lean, the internet is a fickle beast, and nothing looks worse for a company who posts on HN when their technology-oriented site can't handle a few thousand requests per minute. (Or in this case, when a blog claims to handle 4.2M requests a day -- 2.9k req/min)