Instead of "edge", a lot of websites should just have 3 locations (us,eu,apac) with a non geo replicated Serverless database in each region. At least that's what we're building at WunderGraph (https://wundergraph.com/). Edge sounds super cool, but if you take state and consistency into consideration, you just can't have servers across the globe that also replicate their state consistently with low latency. TTFB doesn't matter as much as correctness. And if stale content is acceptable, then we can also just push it to a CDN. Most importantly, you'd want to have low latency between server and storage. So if your servers are on the "edge", they are close to the user, but (randomly) further away from the database. Durable objcets might solve this, but they are nowhere near a postgres database. I think the "edge" is good for some stateless use cases, like validating auth and inputs, etc., but it won't make "boring" services, even serverless in "non-edge" Locations obsolete. You can see this on Vercel. Serverless for functions, server side rendering, etc. and cloudflare workers for edge middleware. But they explicitly say that your serverless functions should be close to a database if you're using one.
Here is my golden setup: Cloudflare Tunnels + All ports closed (except ssh) + bare metal server. You can scale to the moon with like a million active users on a couple of 16 core servers (1 primary, 1 hot failover). You don't need AWS. You don't need Terraform. You don't need kubernetes. Hell, you don't need docker because you know apriori what the deployment environment is. Just run systemd services.
99% of the startsup will never need anything more. They'll fail before that. The ones that succeed have a good problem on hand to actually Scale™.
What we're seeing is premature-optimi...errr scaling.
Edit more context:
For Postgres, setup streaming replication between postgres and hot standby. You need a remote server somewhere to check health of your primary and run promote to your hot standby if it fails. It is not that difficult. Have cron jobs to back up your database with pgdumpall in addition somewhere on Backblaze or S3. Use your hot standby to run Grafana/Prometheus/Loki stack. For extra safety, run both servers on ZFS raid (mirror or raidz2) on nvme drives. You'll get like 100k IOPS which would be 300x of base RDS instance on AWS. Ridiculous savings and performance would be just astonishing. Run your app to call postgres on localhost, it will be the fastest web experience your customers will ever experience, on edge or not.
If you use Tailscale, make sure to keep spare VGA cables or hook up an eth cable to the IPMI port. Tailscale has not been reliable in my experience. There is nothing like straight SSH connection and you can setup `ufw limit OpenSSH` to limit tries to ssh port.
You still pay the latency cost, even though your data travels mostly inside CF's network. It's very noticeable when you're far away from the server, which you will be for most of the world if you sell to everyone. Perfectly fine if e.g. you're only targeting anyone from your region.
This setup is good if you are only serving people in a single continent. The TCP handshake with someone half way across the world is still going to eat all of your latency. You can’t beat the speed of light.
Most startups are going to be operating in a one country. And most requests would be handled by Cloudflare edge except for dynamic requests to the origin.
You might be surprised how fast it would be. And most companies blow their latency budget with 7 second redirects and touching 28 different microservices before returning a response.
All I am saying is don't get fixated on geo-latency issues. There is a bigger fish to fry.
But after all fish have been fried, you’re right. Servers on the edge would help.
What happens if your remote server thinks the primary is down when it isn't really, and you end up with two hot primaries? Is this just not an issue in practice?
In that case, Postgres new-primary would just be detached and will not stream/pull additional new data from the old-primary after promotion. You should also make sure to divert all traffic to the new-primary.
Actually, this might be much simpler with Cloudflare tunnels. So it failover scenario would be something like this:
1. Primary and Hot standby are active. CF tunnels are routing all traffic to primary.
2. Primary health check fails, use CF health alerts to promote Hot standby to primary (we'll call this new-primary).
3. Postgres promotion completes and new primary starts receiving traffic on CF tunnels automatically.
4. No traffic goes to old-primary. Backup data and use old-primary as a new hotstandby, configure replication again from new-primary.
Even better strategy would be to use Hot Stanby as read-only so traffic is split dynamically depending on write or read needs by the app. Take a look at StackOverflow infra architecture: https://stackexchange.com/performance
I wish the thinking that leads to articles like the one in the OP would take physical form so I could take a baseball bat and release my frustrations by beating the hell out of it. So many of my current pains in the ass at work stem from adding complexity to get some benefit that does nothing to make our product better.
the point is the developer audience that is just coming out of college...will not be familiar with postgres.
they would make most of their applications saving data via "Durable Objects" on the edge. Someone just built Kafka using Durable Objects.
Not arguing that DO is better that postgresql. I'm arguing that a lot of the developers wont realise that. Because the DX of durable objects is superior.
C'mon - lets not wrap it up in all that, be real. It's a JSON object and a few JS functions passed as arguments to a class constructor.
I'm not saying it isn't an elegant design, but can we pls not talk about proprietary implementations of a particular design pattern as if they're some kind of industry standard?
You're getting the data format confused with the database engine. Yes, a database might just be storing JSON, but how it's stored and replicated matters.
The architecture I eventually ended up with for my product (https://reflame.app) involves:
1. A strongly consistent globally replicated DB for most data that needs fast reads (<100ms) but not necessarily fast writes (>200ms). I've been using Fauna, but there are other options too such as CockroachDB and Spanner, and more in the works.
2. An eventually consistent globally replicated DB for the subset of data that does also need fast writes. I eventually settled on Dynamo for this, but there are even more options here.
I think for all but the most latency-sensitive products, 1. will be all they need. IMHO the strongly consistently replicated database is a strictly superior product compared to databases that are single-region by default and only support replication through read-replicas.
In a read-replica system, we have to account for stale reads due to replication delays, and redirect writes to the primary, resulting in inconsistent latencies across regions. This is an extremely expensive complexity tax that will significantly increase the cognitive load on every engineer, lead to a ton of bugs around stale reads, and cause edge case handling code to seep into every corner of our codebase.
Strongly consistently replicated databases on the other hand, offer the exact same mental model as a database that lives in a single region with a single source of truth, while offering consistent, fast, up-to-date reads everywhere, at the cost of consistently slower writes everywhere. I actually consider the consistently slower writes also a benefit since it doesn't allow us to fool ourselves into thinking our app is fast for everybody, when it's only fast for us because we placed the primary db right next to us, and forces us to actually solve for the higher write latency using other technologies if our use case truly requires it (see 2.).
In the super long term, I don't think the future is on what's currently referred to as "the edge", as this "edge" doesn't extend nearly far enough. The true edge is client devices: reading from and writing to client devices is the only way to truly eliminate speed-of-light induced latency.
For a long time, most truly client-first apps have been relegated to single-user experiences due to how most popular client-first architectures have not had an answer for collaboration and authorization, but with this new wave of client-first architectures solving for collaboration and authorization with client-side reads and optimistic client-side writes with server-side validation (see Replicache), I've never been more optimistic about the future (an open source alternative to Replicache would do wonders to accelerate us to this future. clientdb looks promising).
Good post. Do you have any resources describing the 'new wave of client-first architectures' you mention? I'm struggling to understand how you can do client-side authorization securely.
Replicache (https://replicache.dev/) and clientdb (https://clientdb.dev/) are the only productized versions of this architecture I'm aware of (please do let me know if anyone is aware of others!).
But the architecture itself has been used successfully in a bunch of apps, most notable of which is probably Linear (https://linear.app/docs/offline-mode, I remember watching an early video of their founder explaining the architecture in more detail but I can't seem to find it anymore (edit: found it! https://youtu.be/WxK11RsLqp4?t=2175)).
Basically the way authorization works is you define specific mutations that are supported (no arbitrary writes to client state, so write semantics are constrained for ease of authorization and conflict handling), with a client-side and server-side implementation for each mutation. The client side gets applied optimistically and then sync'ed and ran on the server eventually, which applies authorization rules and detects and handles conflicts, which can result in client state getting rolled back if authorization rules are violated or if unresolvable conflicts are present. Replicache has a good writeup here: https://doc.replicache.dev/how-it-works#the-big-picture
Replicache has a commercial license but is source-available. Clientdb is open-source, but doesn't seem as mature yet. I'd love to see more open source solutions in this space too.
There with you. We chose cockroach and fly to achieve our global needs in the simplest manner. But to use their cloud multiregion offer, it costs ~3400usd/mo at their recommended specs and 3 regions. Pricey depending on how you see it.
We hope their serverless tier meets feature parity soon.
Yep, Cockroach's dedicated offering was pretty cost prohibitive when I last looked too, and I really didn't want to have to operate my own globally replicated database, so Fauna seemed like the best option at the time.
Really looking forward to Cockroach's serverless options too. More competition in this space is very welcome.
Anyone used Yugabyte before for a globally replicated DB? I use Hasura a lot, which depends on Postgres, and Yugabyte seems like a possible drop in candidate for Postgres, but wondering if others are using it in prod.
How are you finding working with distributed databases? I've worked with Fauna and Dynamo and they are a nightmare from the DX and iteration speed point of view compared to Postgres.
Dynamo has definitely been a bit of a nightmare to work with, but I actually find Fauna reasonably pleasant. You're not going to get the vast ecosystem of SQL based tooling, but the query language itself is designed to be composable, which reduces the need for ORMs (though I do still miss the type generation from Prisma sometimes), and there's a pretty nice declarative schema migration tool that handles that aspect pretty well: https://github.com/fauna-labs/fauna-schema-migrate
Yeah, I mean, apart from missing intellisense, Fauna forces you to set in stone everything, which is not good for fast iteration and prototyping. I understand it's for performance, but things, especially in startups, continuously change.
Let's say the user is in FRA, the database in SF. If the "server" is on the edge, you'll end up with 100-200ms between server and database, while the user has less than 10ms latency to the "edge". If the server does multiple round-trips to the database, it can take seconds until the first byte. If the server and database are both in SF, TTFB will probably be less than one second, as round trips between database and server are almost zero. One thing to mention is that it would be beneficial of the TLS handshake could be made on the edge, as it's a multi roundtrip transaction. Ideally, we could combine a server close to the DB with a stateless edge service.
So, the future of the web might not be on the edge. It's rather: The future if the web will leverage the edge.
