> Imagine if you could just send it the whole “page” worth of JSON. Make an endpoint for /page/a and render the whole JSON for /page/a there. Do this for every page. Don’t force your front-end developers to send a bunch of individual requests to render a complex page. Stop annoying them with contrived limitations. Align yourselves.
Why not just send HTML as a single response at this stage? Sometimes it feels like we are doing web development more complex than it needs to be.
How did we get to the point where simple and effective server-side pages are either unknown as a way of doing things or are considered to be a technical heresy?
Younger developers scoff at us fossils and the olden ways, but this is exactly how this ignorance of classic, tried-and-true, performant patterns results in atrocious complexity and performance, only because "this is how things are done".
It HAS to be a BFF, it HAS to be 15 different microservices and 20 different AWS toys, right? There is no other way? Are you sure?
It seems like we are talking about the whole industry, but it's largely he Node ecosystem in my experience (which is effectively - the industry). It's not like there is not Rails development being done out there, with nothing but a trusty Postgres database.
Ironically, our existing architecture patterns are beginning to resemble J2EE - a bulky, slow, expensive, boiler-plate-laden process where a simple site costs one million dollars and performs like shit.
> How did we get to the point where simple and effective server-side pages are either unknown as a way of doing things or are considered to be a technical heresy?
Demand for interactivity. A real demand - that’s why desktop apps are now web apps.
It can be over applied, sure, but there are great reasons not to serve html templates. The trend is not some massive incompetence.
Incompetence is the wrong word, but in an industry that grows so fast that 3 years of experience is considered senior, many people aren’t even aware that the old ways are an option.
In my experience, the percentage of large react sites where someone sat down and seriously considered using old fashioned server side rendering, but decided on React because the site actually required a huge amount of interactivity is close to zero.
Every React site I’ve ever been involved with in in the last say 7 years was using React because “that’s the way it’s done”, and anything else was just a post hoc justification.
I built a website that was super simple using server side rendering using Django. Then I got a bunch of feature requests that made it less simple and the client-side javascript started getting crazy spaghetti code.
I actually somewhat knew React before this project but besides not wanting to overcomplicate things I was hesitant because I didn't like pure client-side rendering. Then I learned NextJS makes it easy to mix/match client-side rendering and server-side rendering so I just switched to that.
So now, even for a simple website, I'll probably start with React because I don't want to dig myself into a needless hole.
And yes that post upset some people who thought I should use htmx, but React is actually pretty easy and simple. Now most of my websites are React because I know it so it's easy and simple to use what you're used to.
Also, "I did it in React because that's the way it's done" isn't the worst reason because you benefit from that popularity, e.g. some library you need will have clear examples of integrating with React.
I think the frontend and JS community have certainly gone off some crazy rails at points, but I also think there's a popular sentiment on HN exaggerating how off-the-rails things have gone, usually expressed by people who don't actually develop many modern-looking websites. Most of the trends have gotten popular for somewhat rational reasons, and even that stuff that has gotten a little crazy like an explosion in dependencies, is really just about tradeoffs e.g. lots of dependencies cause lots of problems but also enable code re-use so it has pros and cons.
Software engineering isn't a hard science, it's impossible to be 100% certain in one of these debates. You could be right and I could be wrong.
But I've been doing this for a long time, and that's what everyone says. I've heard this argument at least 100 times when someone is trying to defend their decision to build something overcomplicated. The number of times they ended up actually being justified is so small that I'm very suspicious of this argument.
I’ve had the same experience of going fully server rendered to keep complexity down and ending up with having to turn down requests for more interactivity to prevent the code from turning into a spaghetti mess. Maybe not necessarily a bad thing to have less interactivity but it’s definitely a trade off to be very aware of.
I really enjoyed your blog post! (You have a broken link to a time zone post, FYI.)
If you're not expecting to need fancy client side stuff, does next.js give you as good a server side development experience as Django/Rails/Laravel? At that point, it would seem worth it "just in case", as you suggest, but in the past when I looked at it, the SSR stuff still felt a little cobbled together and experimental. I would love to hear from someone with firsthand experience switching, though!
Thanks for the kind words and the heads up on the broken link.
I think Next has a great DevEx, but I'm just using it for some projects as a solo dev without a ton of traffic so perhaps there's some rough corners I haven't experienced yet.
I do think they are going down some risky paths where they do things like rewrite your code files depending on whether you're on server or client, and in general with "framework-defined infrastructure" where if you write certain functions the framework "knows" that it's supposed to be on the server or in a background job or whatever. But so far I've gotten the benefits without the drawbacks, though I might be jynxing myself.
There's also an interactivity requirement (as in, this is impossible without interactivity) and an interactivity desire (as in, UX folks want an interactive feature).
So many websites now pretend they're in the former category, but would work just fine if they implemented a less-interactive form. That's how it was done in the old days! "No, you can't have that. Here's the best we can do." "Oh, that's fine."
But I expect more of what's driving complicated backend/frontend split design these days is shipping the org chart (in companies large enough to have frontend and backend teams) and technology-by-consultant (where the right technology is whatever you can get a consultant in).
If you want to build a moderately complex interactive experience that doesn’t suffer from roundtrip latency on every single interaction, you need to recreate the interactive parts of DOM in JavaScript anyway (ignore wasm), else you need to do imperative updates which quickly become a nightmare. jQuery-oriented development is a classic for sure, doesn’t mean people want to go back to it.
Because JavaScript rules web dev and the JS community has pushed so many mainstream ideas that have turned out to be duds. It’s a major echo chamber due to its size relative to other web dev communities.
I also partly blame the careerist obsession around always learning new technology, because it’s becoming obvious that a lot of the “new stuff” over the last decade created more problems than solutions. Don’t get me wrong, learning is an important part of the job, but we’ve created a culture of constantly pushing new technology to pad resumes and feed our insecurities around irrelevance instead of solving problems.
Hype, which the web/JS community seems particularly vulnerable to, also plays a big part. Yes proven tech is safe and reliable, but choosing it is so much less fun than getting swept away by the tide of whatever the newest "revolution" is. Don't you want to be part of the wave?
Personally, not for the day job. I love experimenting with new stuff in my free time but when the pressure is on to deliver for the business, I want to go with what I know works.
It seems to me that the more people use <thing>, the less evidence of quality of <thing> is required by the people using <thing> - everyone just assumes everyone else knows what they're doing.
It becomes something akin to a religion - after all, 2 billion Christians can't be wrong, can they?
> everyone just assumes everyone else knows what they're doing.
It’s not just that. There are practical reasons to choose the bandwagon, even if you know it’s not the technically optimal approach.
If you’re planning on hiring junior and mid level engineers, for example, it’s a great help if they already know the technologies involved. Your hiring pool for React is a lot bigger than for a more obscure tool. Additionally, the popular tools also tend to be stable and long lasting, with good support.
So, there are compelling business reasons to choose what’s popular, even if you aren’t making assumptions about the technical quality just because its popular.
Eat shit - billions of flies cannot be wrong. (cit.)
More seriously: once a bit of tech becomes mainstream, it becomes politically easier to use it. Your manager has not heard of <cool tech>, but he did read about React in some business magazine. So when the choice is between React and bheadmaster's Funky Funkness Framework, he'll go with React.
I'm torn. On the one hand, there are many websites I hate interacting with because they are unnecessarily structured as JS clients exchanging data with a remote server via a network link. On the other hand, it's easier for me to develop libraries and CLI clients of my own, these days getting quite far with Firefox dev tools' network view and "copy as curl," and only occasionally having to actually read the code in those JS clients. In the old days, I would have to resort to some ugly and comparatively brittle page scraping.
This new world sucks for the interactive user in a browser (which is me often enough), but it's great for the guy who wants to treat a website as just a remote provider of some kind of data or service (also me often enough).
> How did we get to the point where simple and effective server-side pages are either unknown as a way of doing things or are considered to be a technical heresy?
We are not at that point nor we were ever at that point, or close to it.
Server-side rendering has been a hot topic in JavaScript framework circles for over a decade, and React famously solved that problem so well that it is now a standard, basic technique.
Also, just because you throw the same buzzword around that does not mean the problem stayed the same. Reassembling a javascript-generated DOM with a coherent state is not the same as sending a HTML document down the wire. And you know why people started assembling DOMs with JavaScript? Because sending HTML documents down the wire was too slow and did not allowed for any control over aspects that dictate perceived performance.
The current SPA paradigm was adopted to solve a number issues that engineering teams were facing. Your sentiment is one I'm seeing a lot more as of recently, which leads me to believe we're approaching the end of the current web development paradigm. There's too much time wasted writing _glue_ when developing SPAs: database queries, back-end controllers, data serialization, network requests, front-end state, shadow dom, etc. Lots of frameworks coming out that essentially remove the need to write glue code. I expect this trend to continue until we reach the point where you are mostly just coding a few things: schema, non-generalizable business logic, and presentation.
Blazor Server, while definitely not the right thing for all applications, is the best solution I've found to this.
You write all the code in one language (C#), and it streams the changes to the clientside as needed automatically. It means you can call the database in your 'frontend' HTML layouts directly and gets rid of all the glue code. You have no serialization problems because you are using the same classes everywhere, and the component structure keeps things very clean.
I would say I am about 5x more productive in Blazor Server than any other frontend technology. You don't need to write APIs at all. It's just like writing static server side rendered pages but you can do dynamic stuff. It is honestly like magic.
It does come with some huge downsides though, in that it needs to keep a websocket open to stream all the DOM changes from the server, and this also has memory/performance issues on the server (needs to keep state on the server all the time). Which basically rules it out of anything that needs to work over intermittent connections (mobile), and very high scale stuff (probably could be done with enough server RAM, but it's not the right tool for that).
However, it's perfect for boring line of business applications that will only be accessed on solid internet connection, which is many of them.
There is also Blazor Webassembly which removes the websocket part, and lets you code frontend and backend in C# still, sharing a lot of classes, though you do need some glue code to connect the database. I've heard people are having good results with gRPC for that; but haven't looked into it much (blazor server works fine for most apps IMO).
Not super famliar with blazor, but something like this sounds like what I would consider the halfway point to the next paradigm. I think you have to go further in simplification of the back-end as well.
Hadn't actually heard of that. Might be similarish but Blazor uses standard html/css/JS and just sends DOM diffs down the wire. Don't think at first look Vaadin works in the same way.
Ah ok. So the server keeps track of what the client should look like and just keeps it up to date?
If so that sounds like a very nice model that would make it a lot easier to get back to just using simple HTML and delegate the complexity to the framework or server process. Are there other projects beside Blazor that use this type of approach?
As far as I can tell, the SPA paradigm solves exactly one issue: Whiny users saying "but it flashes and reloads the header and navigation between page views and desktop/mobile apps don't.".
Since HTML never had a proper way to compose content from multiple sources-- frames being as close as it got-- we have to instead punt this, like so much else, to JavaScript hairballs to glue it all together.
It is also touted as a way to grow engineering talent pools by subdividing into frontend and backend teams. This is not super convincing to me in a context where the only frontend is HTML given the enormous amount of extra work that a SPA takes to develop. It is more convincing in a context where there are native applications in the mix that would benefit from an API. So what will a future that involves both MPA-style frontends and APIs look like? I think one possibility is that backends will handle data in a more declarative fashion. In theory, doing so would allow a lot of the "glue" to be generated, and ultimately allow more flexibility on the frontend.
I think it's the difference between generations of people for whom JS was an extra thing to enrich HTML documents, vs. younger generations that see HTML as merely a rendering target for JS.
And we got here by having javascript as an embedded scripting language in things like access control management. "I'm already using javascript on the server in my polkit config!"
"A BFF layer consists of multiple backends developed to address the needs of respective frontend frameworks, like desktop, browser, and native-mobile apps."
A BFF is just an API gateway that is tailored to a single frontend. It's especially helpful if the frontend is a native application as opposed to a web application, hence why it was a pattern championed by Netflix. If you have some native client out there and you can't guarantee that the user will update it, it's helpful for it to only talk to a single service so in case you want to evolve your backend architecture, you can update the BFF implementation and the existing native app won't be broken by your backend changes.
Having a native app out there in the wild on customer computers that is coupled to a bunch of different microservices is hell for when you want to change your backend without breaking your customers' experience and forcing them to update.
The backend-per-frontend thing makes sense for Netflix. They're huge scale like I said, plus they've been around a while, so a lot of backwards compatibility issues will arise. I can't see that being a default pattern to jump to, though.
Backend that handles all the microservices so the frontend isn't doing that job, sure. I didn't even consider doing it differently than that.
Huge scale typically leads to the opposite: more backend generalization, not catering to a specific front-end. That's why Facebook built GraphQL — because they run on anything, like fridges, TVs. That's specifically an attempt to avoid tailoring a backend to a frontend. The article argues that at small scale you shouldn't try to build a generalized backend for any imagined frontend. It's exactly how you would get unnecessary performance and complexity bottlenecks, when you really just need to serve your data the 1 or 2 ways that your front demands.
Yeah, our org has a big problem with over-generalization in our internal services that I've been pushing back on. Due to the large size of our org, they're under the false impression that our services are large-scale, but they're actually very small compared to anything external-facing.
When a partner team filed a feature request on our team, and I gave them an API that just does exactly what they need, they were like "that's it? No 1GiB graph response to DFS through?"
Most of the time time BFF just means that instead of having the frontend call endpoints A, B, and C. It calls a single endpoint D, which calls A, B, and C and handles aggregating and transforming the data into the specific form the frontend needs.
I don’t find that to be an accurate representation of BFF. I’ve used BFF with a browser FE to simply weave several microservice endpoints into a tailor-made api for the frontend. This reduces the burden on the microservices by removing the need to make many endpoints for clients and helps reduce network calls on the FE
We have a boot strapped and profitable app with thousands of customers written in Django that has been operating fine since 2009 with new entrants raising and spending millions to build fancy BFF/Micro Service solutions, hiring sales people, losing money for years, only to publicly state they have finally gotten to 300 customers.
Feels like the author misses the point of an SPA. If you have website where every "view" is a /page/a etc., SPA might not be the best choice. But in an SPA i can be dynamic, and trigger data retrieval/sending without route changes.
A very basic example is if I have a checkbox that allows the user to subscribe to push notifications. On click I sent a xhr request to the backend POST `/subscribe/1234` which registers the subscription. Next time I'll check `/subscriptions` or similar to see if the checbbox is in "checked" state (i.e. we are subscribed). This is basic functionality which requires a concise REST/JSON API, and has nothing todo with your page/route layout à la /page/a, /page/b etc.
We do this for SPA. The POST can be supported just like you explained. Then you can either reload the "page" to see that the checkbox is checked, because backend will include that as part of the page, or (as an optimization) reload just that checkbox if you created a specialized resource for it. The entire page is still there for you to fetch upon transitions.
Among pages, you will still need a few individualized resources sprinkled around for optimizations/reloads/fragment-paginations. The difference between that and a complete API is that they will be entirely design driven. If a table needs paginating, and is built with data from multiple resources, you will not provide 2 resources, and expect front-end to paginate and glue them together. Instead, you provide a complete paginated resource for this table, where data is ready to be displayed as-is.
As a user, I can think if a couple of reason immediately:
- I don't like page (re)loads. They are usually slower and more likely to fail, compared to a lightweight request. Especially in scenarios with a bad connection
- If they fail, it's harder to retry, I see a connection timeout page. With a SPA I see an error message, potentially with a retry button. Or even better: the SPA retries for me a few times.
- I can continue to see all the rest of the page while the action is running
- I can potentially start other actions in parallel
- I prefer to not lose any progress of things on the side, e.g. text-boxes where I already entered/changed some text
- It's easier to inspect what goes wrong by looking at the network tab. Okay, most users don't do that, but for me it's still a pro
There are also advantages, but I think nowadays the cons outweight those for me.
Yeah, sorry, but I just don't agree with the spirit of the comment, and I think it's thinking like this which leads programmers to over-engineer web applications, which usually harms user experience and wastes company money.
Absolutist statements like "no, you should not do a full form post just to toggle a checkbox" are just silly. There is no minimum bound beneath which a standard web browser form submission doesn't make sense, and writing extra JavaScript code to not only manage an asynchronous network request but also handle subsequent behaviours for success, failure, timeout, etc., is additional complexity which incurs additional cost and a greater potential for system failure.
Sometimes these implementation details and associated costs are necessary, but in most cases they aren't, and it's not an ideal perspective economically to default to the more expensive and complex implementation, especially for dubious benefit.
Ok. I don't see it that way. Most SPAs on the internet should never have been SPAs and it's usually poor judgement which leads a project in that direction. That poor judgement includes things like the characterisation of full page reloads as overly cumbersome and asynchronous requests as being lightweight. It also includes the expensive decision to try to reimplement the native browser behaviour that you get for free, as the commenter alluded to by suggesting asynchronous requests can be programmed to show a custom UI for error messages and use custom behaviour for retries of failed requests.
I believe I have indeed taken the time to understand what people are saying in these comments specifically and on this topic more broadly. And I just don't agree with the opinions presented. I think they're silly, in the same way that microservices are almost always adopted for silly reasons. The popularity of either approach doesn't negate their silliness.
> Absolutist statements like "no, you should not do a full form post just to toggle a checkbox" are just silly
That is because you didn't read or accept the frame that I set initially. I was very clearly refering to web apps that are SPAs (not websites!). Within that context, that statement is less absolutist and still true in 95% of the cases.
Very sceptic that RoR can obtain/interact with the browser push notification API as I stated in my example. You'd have to come up with glue JS code yourself, and that is where it gets messy. Pushnotifications just being one of many APIs that are client-only (and again please, if you don't need to interface with those APIs, then you might not need an SPA in the first place).
Just because "remote forms" have forms in their name it doesn't mean that they are conceptually the same thing. They are not, so obviously they can achieve those things, but they are built with js.
Having form elements automatically update the back end is extremely annoying UI imo. There's usually no indication that it's actually doing that, and if there is I generally don't trust it.
Yeah, this goes back a couple of decades where "page a" was just "page-a.asp" or "page-a.php" and you did everything to do with that page in that file. Simple, and you could mostly make changes to page a without worrying that you'd break other pages.
Need page b? Copy page-a.asp to page-b.asp, gut the page-a logic, and put in the page-b logic.
Before you get too nostalgic for these simple days, do we rememember why we stopped developing web pages that way?
Because you can't have a lot of itneractivity with a nice responsive UI like this if your network has high latency and low bandwidth. So we pushed a lot of logic to the browser. Now it turns out it doesn't scale well, so maybe we should go back to copying page a to page b and calling it a day, same way we came back to static linking once disk space stopped being limiting factor.
I’m a systems engineer and SPA’s are a superior architecture to deal with from a delivery POV. It is very difficult to cache a dynamic web app with a CDN. There are ways, like SSI, managing variants, Purging, and so on, but a huge pain. And if you don’t have a sophisticated CDN and are using something like Cloudfront, forget about all those fancy features.
SPA? I can serve the whole thing with static files and remove 90% of the traffic from my web server. The API can be served from a different path from the static files. API requests are super tiny and more tolerant to fluctuating network conditions, and you can hide a lot of that with Javascript.
I have a hobby project that's very static: a website for viewing webscraped snippets of writing from another site. I hosted it on Github Pages. I didn't want to use a static site generator which would've generated easily 10,000 files and with all the duplicated code would have put me above GitHub's 1GB limit for Github Pages. Similarly Cloudflare Pages has a 20,000 file limit.
So instead I combined it into various JSON files. Now I have a SPA downloading the JSON as necessary, and don't need any API/backend/database. I'm considering moving to sqlite now there are official WASM builds.
Exactly. Even non-SPA frameworks (such as NextJS with static builds) mean you can compile all of your frontend code to S3 (where it's a negligible cost) and only pay for the API instances.
Because for the last 10 years, “code schools” and boot camps have taught JavaScript/Typescript with React as “programming” where everything outside of your immediate dom is an api call. One not worth knowing about since it’s “backend”. The reality is SSR has never been a bad idea, we started that way. Now we are seeing reimplementations of it in various forms to “solve the api call hell” that was self induced. Just like the front-end bundle size problem was self induced.
True, you could insert something in the middle, that generates a page depending on the user-agent.
But then it requires extra SSR logic.
But on the other hand, you can have a iOS or Android based app that depends on the same data. So the browser is not the only client in general, especially nowadays, I think?
That's the crux of the question. The BFF approach says that you remove logic from the client and put it in a server-side component. Not that you add extra logic. Then you might find that you need to copy that server-side component sideways for a new client, if the view layer in that client is sufficiently different to what you already have. The browser can be the only client of its BFF API; the mobile apps can either speak to the same API or have their own.
Dramatically more important than any theoretical, abstract consideration in deciding whether this can work is what your teams look like. The client team should almost certainly own the BFF API, which has implications for skillsets and maturity.
Nothing prevents the main response from being served as regular HTML and subsequent XHR request for some parts of a page being JSON or some such data format. In fact I bet that sending server-side rendered HTML is faster than sending a full structured page as JSON which has to be decoded, parsed, and then converted to HTML.
I run a popular app (native ios and android) and a web version. I find having A single json serving api much easier to maintain. The frontends are isolated projects. The backend is just a simple layer with static data served as json. If you would have static html it means you need to update all static files whenever a ui change is needed. Makes it overly complicated. Caching becomes also a challenge and the so does the performance.
Currently just running a few Hetzner instances to serve 50k requests per second. That would be really hard with a server side rendered html.
That's true. But that's only really a concert if you're building an SPA. If you're not, having multiple pages is generally faster than running some js framework to reload parts of the page. Plus that you get the benefit of every state combination having it's own URL. You don't need to write any custom state reconstruction code.
This was the original design of REST as applied to the web. It was explicitly designed in such a way that it was forbidden to reload parts of the page. This makes it so that every state has it's own URL and you therefore can link to every state. Deep linking if you wish, for all of the web.
The is exactly what HTMX does and it works: allow returning a HTML response and statically annotating a part of the DOM to be replaced with that.
Granted, this requires JavaScript on the front end, but it's a framework that can be shared between different websites, so you're not writing any custom JavaScript for your site.
Only if there’s an update - htmx will encourage you to render full html pages but still gives you the ability to do in-place updates which is what most folks think a JS front end with a json Api backend and client side rendering is needed for.
The first 3 sentences of the article are implying that the whole thing is written with "if you must" attitude.
That said, a lot of companies have front-end teams, and those front-end teams commit years to building React components. So even if you render server-side, this is how you'd put data into those components. I'm a backend dev. I don't like some of what frontend is doing, but you gotta work with people, right?
Because you want to paginate. Update the page piecemeal based on an event from another user. Because you want to use the same data on multiple pages, and want to write the authorization and filtering logic once.
Got to pump CV with frameworks you know? Promotions don’t write themselves. Bonus point: job security. And when you leave who cares what poor shmuck will have to support it?
One extra thing I have confirmed after writing this article: it's usually a bad idea to reuse back-end data in multiple places on the front-end.
If you think about functions or object constructors in general, it kind of makes sense. One of the most important architectural practices I've ever discovered is: don't pass in the parameters that you have, pass in the parameters that the function needs. So if you have companyName, and you are constructing a page that has a title, you shouldn't pass it companyName: companyName, you should pass it pageTitle: companyName (parameter name: parameter value). It's crucial that the parameter is named after what is needed, not what you have. This is kind of the essence of what the article is suggesting.
If your front-end is reusing a lot of fields from back-end, it's likely that you're passing it what you have in your database/resources, and not what it needs — fields for constructing the UI, configuring the views. Moreover, I rarely see front-end going through the exercise of defining exactly what they need. The article encourages more of that.
Once the front-end starts reusing the generic back-end fields everywhere, you lose track of your use-cases. Everything you ship from the back-end becomes potentially important for mysterious reasons that not even front-end can easily understand anymore. Front-end already built complex dependency trees based on these generic fields. If you actually untangle these trees, it might turn out that your front-end can only exist in 5 different "modes", but you can no longer see that in the tangled mess. You can no longer streamline and optimize these 5 configurations.
So to summarize: pass what is needed, not what you have is a good general architectural rule that's also applicable between back-end and front-end.
This and the article strike me initially as completely wrong, which makes it very interesting. I still want a clear separation between the "model" and the "view", and I guess I tend to assume that this corresponds to the front-end and back-end. Maybe that's a poor assumption.
What I see here is that the "view" is split across the front-end and the back-end. Or in other words, the back-end contains both the model and the JSON API as a view. I think this is what is meant by "I suggest you stop treating your frontend as some generic API client, and start treating it as a half of your app."
I still want a clean separation between the model, e.g. companyName, and the view, e.g. pageTitle. Somewhere, there needs to be the explicit logic (in the back-end part of the view) that says that the the pageTitle for page A is the companyName. It's okay for many different views to "reuse" fields from the model, but each view should have its own front-end and back-end components. That the view is split across a front-end/back-end divide through the nature of the web-stack doesn't change the fact that each view should remain distinct from the other views.
So I can agree in the sense that the alternative where you create a general-purpose JSON API as a model and view in the back-end, and then essentially code up an additional model and view for each front-end page/element with logic to convert the API data into the necessary content is a waste of time.
A backend model is usually just called the model as it’s modeling the data domain (sometimes called ‘domain model’). The thing you pass to the frontend is the ‘view model’ which is often entirely different.
In some CRUD situations though, they can be nearly the same and you can convert the domain model into a specific view model with some kind of adapter (how people do this varies a lot). This adapter will take the end user security level into account, for example. Also, to avoid sending lots of useless data for a list, you can have a list view model. But if you do that, be sure to either keep the two in sync, or you have one and keep track of whether it’s partially or fully hydrated.
I agree that mvvc sounds exactly right, but there is still the question of whether the view model is created in the back end (which is what the article is advocating) or in the front end (which is what you have to do if the back end serves the model like a general use API).
The MVC or mvvc or whatever paradigm is somewhat orthogonal to the technical back end/front end.
Arguably, the opposite is true. The view is more washed out across the stack if back-end supplies generic fields, and front-end decides how to use them. Now you no longer know who is responsible for which decisions, and have to look for them everywhere in the stack.
If back-end provides the entirety of data to build the page, then you limit front-end decisions to UX based on the specific data, and no longer allow them to pull in new functionality or foundational business logic over to their side.
MVVM definitely came to mind as I was writing my comment. The model is clearly in the back end, the view clearly in the front, and the view model contains the explicit link between companyName and pageTitle and can live in either. Here the suggestion is to put it in the backend and serve it as JSON.
I'm suggesting to pass via JSON what we used to pass via HTML back when front-end was templates on the server. I still prefer those days, like I alluded to in the first paragraph of the article. It used to be: write your template by inserting database values into it. Now I'm saying: instead of a template, come up with a neat json structure that kinda represents that template, and insert your database values into it. Name the fields after what page needs, not what you have in the database (following standard practices of object construction). I'm not sure what I might be reinventing, or what it might've been called, because I'm just trying to explain a practice that saves time and reduces complexity in simple language. It's interesting if it already was a thing with a name, but doesn't really change the point.
I think if the back end provides generic fields and the front end acts as an independent consumer of the API, then you essentially have two apps each with their own entire MVC (or whatever paradigm you are using), one on the front end and one on the back end. So I wouldn't quite say that "the view is more washed out across the stack" but that you have two views (and two models).
Agree with this. Of course because front-end still gets to build business logic, and back-end still ultimately decides the basis on which to build it, the decision making is spread more equally between them. Perhaps calling it "a washed out view" is not strictly appropriate, but worth acknowledging that the front-end is at the mercy of what back-end decides to provide it.
Seems like you couldnt have front end engineers with this philosophy. Or you could but it would be very unproductive. Because theyd need backend changes for every API call.
While this may be true to some extent, I think front end has plenty to deal with even if you do ship them the exact data they need.
Primarily, this philosophy encourages you to maintain a strict connection between needed use cases, and the code that enables them. Once that connection is severed, your code expands beyond the required use cases, and you end having to maintain support for phantom/theoretical use cases.
This approach would also be a nightmare if you ever decided to refactor the frontend. You'd be forced to do double the work since failing to update your backend would probably result in nonsensical legacy naming... I'd also dread to think what would happen if you also used the backend to serve a mobile app. You'd end up spending 40% of your time trying to figure out what to name fields before eventually going full circle and reverting back to logical generic names :|
And the moment your SaaS is growing, and your customers request access to your API:
Instead of just implementing ACLs/permissions on existing APIs you have to develop and maintain two APIs,
one for the customers (that will get outdated with lot of missing features and be less battle-tested)
and one “real API”.
Our public API is fit for the use-cases of our big customers that want deep integrations, and our frontend’s API is able to handle our specific workflows.
We have a bunch of shared backend code, but the endpoints that our API customers use are very specific. Sometimes we build custom endpoints for a single customer which would be useless for our frontend/services. Keeping them separate allows them to gain needed functionality without trying to bolt it on to existing internal APIs, and cluttering it all into one messy blob.
Of course our internal API is a mess in a bunch of other ways but at least we don’t have to deal with the public parts making it even messier
Redesign: I've responded to the redesign argument in another comment. It's easy to find if you search the page for "rare event".
Mobile app: this is addressed in the article.
Naming: Usually most naming for different parts of pages is handled by the design(er), but even if not, once you have an approximate naming convention for parts of the site, naming becomes easy. And the stakes for these names aren't that high, because you are only scoping everything down to a single page, so global consistency and precision in naming is not that imporant. It just needs to be enough for someone to understand what it most likely refers to on this page.
Ignoring my general thoughts on full stack engineering for the sake of brevity—this works okay if your FE and BE have enough overlap in necessary skill set (e.g., same language) but in my experience when you need to hire a senior Rails+Vue3 developer or something you’re going to be tearing your hair out.
Yeah. This is suggesting to me that the "model" live in the back-end obviously and the "view" is separated across the front-end and back-end. I think that does sound more straightforward for many cases. So working on the "view" requires being not exactly a "full-stack" engineer but at least being able to make changes both on your front-end part of the view that makes API calls and back-end part of the view that serves them. I think that makes sense. You can still have fully "back-end" developers that handle the database and actual server, and fully "front-end" developers that handle ui design and content.
Every API call was so form fit to the UI that nearly every UI tweak needed frontend and backend changes.
It didn’t need to be this way (there was a lack of engineering skill and leadership), but even past the surface level much of the backend code beyond the API interface also became form fit to the specific use case meaning many times seemingly simple changes required near total reimplementation of all of the business logic and other underlying code.
End of the day, showing the same data or exposing the same flows elsewhere in the application required nearly a total reimplementation on the backend. This created an explosion in size of the codebase and complexity.
This whole idea seems to basically be going back to RPC-based APIs and all the drawbacks that led to everyone kind of collectively agreeing REST was better (even if most people only sort of half implement it).
Our approach, staring down a roadmap of projects that basically involved exposing the same functionality over and over with different pictures in front of it was to steer _hard_ the other direction.
We sat down and put together an API specification that laid out the general shape of an API. Not the specific resources, but for any given collection/resource how it should be exposed. We found common “tough” use cases and figured out up front how to expose those RESTfully and documented the patterns to maintain consistency. We put down a framework in code to make compliance with the spec the “easy” path for most cases.
Then we approached the rest of the design from an API perspective, not a frontend or backend perspective. For any given flow/operation/etc, what would a good interface look like for this. (This is very similar to figuring out what the interface for a module/class/etc should look like, given its responsibility, before writing it.) Then we built it.
The end result has been that more and more frequently requests and even entirely new projects require _no_ backend involvement. Often the FE team doesn’t even need to ask any questions—given any endpoint that exposes a collection of resources, they know _exactly_ how to work with it because they’re all the same.
When new business rules or requirements come in, we no longer need to apply them in 5 or more different places across two codebases. And we’ve still maintained a good separation of frontend and backend duties to the point our FE team is busy but kind of bored—they’re only really concerned with the view/display logic at this point.
Company history is largely lost to the mists of time, but the API routes were all prefixed `/v5/`. I can only imagine how much further along the company would be if they’d spent the last decade building on top of a foundation instead of rebuilding the foundation every two years as they have been with the RPC model.
> showing the same data or exposing the same flows elsewhere in the application required nearly a total reimplementation on the backend. This created an explosion in size of the codebase and complexity.
I'm trying to understand how this is possible. Couldn't the backend logic that contributes this response fragment in one endpoint be extracted and reused to contribute the same response fragment in another endpoint?
Was backend not modularized and reused?
The way I understand your approach, you ended up building and maintaining CRUD resources similarly to how they would exist in a naive public API, but the difference is that you spent extra time to build more meaningful, rich, business-level resources than serving nearly raw database records.
This is probably a good compromise that isn't disagreeing with the article as much as you seem to be implying. The difference between your approach and the article's approach is a small side-step. You either let the front-end fetch your rich business-level resources separately, or you package them into pages and serve them in a nice single package. Or you can build pages as per the article, and extract these rich resources that you display in those pages into separate endpoints for some additional flexibility. Either way, you end up with rich use-case driven resources, packaged differently.
It's an example of the general rule to name/implement functions according to what they do, not how they're used.
Stated another way, when building a tool, don't let information about the incidental context leak into the design of the tool itself. It should be designed to serve its intended purpose independent of when, how, where, or why it was built. This way, it won't break when that context changes, or when it's used in a new context.
I was nodding along until this part: “It should be designed to serve its intended purpose independent of when, how, where, or why it was built.”
This is impossible and in many cases undesirable. As long as the purpose and behavior of the function is clear, it is OK if it has contextual roots. Actually, no, it isn’t just OK, it is the way it must be.
Take one aspect of a sorting function for example. Should it be stable? Or not? This decision is rooted in the context. To try to make the function independent of context is impossible and unwise.
Whether it's stable or not is irrelevant, if it's "stable_sort()" then it always should be, if it's "sort(stable: bool)" then it could be. It shouldn't depend on why and when I intend to use it. Most importantly it shouldn't suddenly become stable when I pass "sort(companyName=Walmart) because that's what we needed to do for Walmart but none of the others. That's outside context leaking in and a big mess to maintain.
The discussion here is rooted in how we understand ‘context’.
Your comments about stable sorting are well and good, but they don’t disprove what I was trying to get across with my example. Why a ‘business logic’ function uses a stable sort (or does not) is contextual.
I understand your point, but do you understand mine? This is just one example. The idea for mutually beneficial communication is this: one person strives to be charitable and understand another’s example as a way of understanding their point. This is not a skill necessarily taught or nurtured in the field of software engineering, but it should be.
Most of us have the deductive/analytical skills to use an example to prove our point. It is actually harder to take someone else’s example and see their point. It is harder empathetically and computationally; it requires some inductive steps. (I think there’s a connection between empathy and generalization/induction, but that’s for another time.)
My comments are obvious — almost tautological — if you think epistemologically. Programs and functions are designed to solve problems in context. You can’t get around this fundamental truth. Philosophy has been around longer than programming; in fact, it is one of the historical roots of mathematics and programming. Discussions of good design, which may involve parsimony, orthogonality, and more, have been discussed at length well before software existed. We’d be wise to not ignore lessons already learned.
Such is my experience, having seen and learned a lot from a lot more than software. Claiming that something is context-free is a bold claim. Pure mathematics and logic are probably the only areas than can make that argument credibly.
You are basically saying we should take a name that conveys info about its content (companyName) and change it to something that contains none (pageTitle).
I can tell if a field is being used for a page title by looking at the html, I can’t tell what it contains unless it is named well. Additionally if I ever want to hop into the backend my time on the frontend will have given me 0 context if you are renaming all the fields.
Also if you want to use Typescript this would prevent a bunch of automated type generation you could do.
They're suggesting using the pageTitle parameter for the title, and being explicit about how things are used. If you pass a companyName parameter, it is quite generic. You just happen to use the companyName for the title, but you also use it elsewhere. Someone might want to add some additional info to the title. Their solution will likely be a new parameter and then concatenate these values together. But if you just thought about being explicit to begin with, then you'd already have the functionality to do that.
The idea is to write things as standalone components. Which is a pretty universal programming strategy.
Like most things, it depends. If you have a generic math module, you of course want generic parameter names. It would be weird to have a differential math library with a bunch of random terminology from another domain mixed in.
If the page only makes sense if a company name is used as a title, then of course use companyName as the variable.
The result with this strategy is that you may need the companyName and end-up at some point with code that does back:
companyName = pageTitle
or worse:
companyName = pageTitle.split(‘ - ‘)[0]
Because there is a moment when you will need to get the companyName again for some comparison or some whatever logic (e.g. is selected account profile current company ?)
This feels very confusing.
Instead pass companyName and keep it that way until the very last moment that you display it on the page (e.g. when rendering the view/template)
The whole point is that when you need companyName for another thing, you should create a parameter `anotherThing` and backend will set companyName into it as well. It's designed to make it very weird to _not_ do that, so that you don't do that.
You guys are each partially right. What you need is MIDDLEWARE.
On the one hand, you have some data coming in format X.
On the other hand, you need the data in format Y.
What you need is to define how to map X to Y. And then when you request Y, a way to indicate what parts of X you really need, so the code getting X can optimize, if it needs.
In my experience, the real issue is how to cache the X - should we be caching all of it depending on the Y, or should we be caching parts and understanding how they fit together. I have had problems with updating caches, and I am looking for solutions!
I think what you are looking for a is "(pure) functional reactive programming".
The problem with caching is that it is very often highly individual and it usually comes with drawbacks, i.e. you win performance here but lose it there or you win performance but you lose consistency. (related: CAP theorem)
So I doubt there is any "generic" solution that will or even could make you happy. But if you find one, let me know. :-)
Well, in a single-threadsd environment you almost can have consistency, and you can cache the latest results. What I am saying is that if a result consists of X1, X2, X3 but you don’t need all of them for a certain request, then you wind up caching X1, X2 one time, and requesting X1, X3 another time, what to do with the cache of (X1, X2)? Currently I just update the caches which are subsets (X1) of a cached response, and do it with manual individual code. I hoped that I could have some sort of dependency graph. Maybe reactive programming, yeah. But that would essentially be some sort of declarative language, to run pure functions which still have custom code. Hmm
Single threaded just means there is no parallelization, but there can (and usually is) still concurrency, which is what reactive programming is about.
I think reactive programing solves the "dependency graph" problem to you in the sense that you just have to "change" one value and then all dependencies (and their dependencies) update. You still have to define the relations though.
I don’t know about this. I do think people tend to prematurely optimize by over-generalizing their APIs instead of building what they need. But this seems like it swings too far the other way.
This is really timely for me, thank you. I’m ignorant when it comes to web dev best practices (data engineering background mostly) but find myself suddenly having created a couple small backends for some of our internal (non-customer facing) tooling, and running into basically all of your questions.
One of these I struggled with right away was “how do I ensure the front end and back end have consistent data sourcing and validation”, e.g. for various forms, dropdown selectors, etc. I have found surprisingly few resources for general patterns here. So I did your anti-pattern of having an endpoint for each little thing (e.g. /api/v1/country-codes or something like that).
This has felt… ok at best. One challenge, perhaps entirely imagined, is coherent namespacing. Part of the backend is also a public (internally) API. We auto-generate all our docs with swagger, and I was finding that the few meaningful endpoints I had were diluted by the mess of individual endpoints for the frontend. So I threw these under /api/v1/config. Feels like a junk drawer but at least it cleaned things up.
I like the idea you’re proposing here for having one API per page, I think it makes a lot of sense. It also eliminates the “I guess I’ll add another endpoint” and changed it to “let me add another field”. Feels more granular and scoped. I will be giving this a try!
Because I’m otherwise uninformed to pros/cons here and don’t have the benefit of experience to fall back on, what are the alternative suggestions for best practices for these types of problems? For context, my app is a single backend dev (me), separate frontend team that tasks on as needed. Backend dev resources are unlikely to expand beyond me for the 2-year horizon.
I also have all context on vision for direction of the web app, and field all user feedback and requests. Front end team has no context on user needs, and does not keep any roadmap for the app. So it seems like the above recommended pattern could make sense in that I could drive more front end work from the backseat, so to speak.
1) basic CRUD API for your objects (eg DRF generated model-API in Django). Really easy to build. Client is assumed to have some magic knowledge (eg which country code style you use). Ideally as much of the smarts as possible is server-side, but early on you can compromise on this.
2a) for reads, notice you need nesting / joins, add nested fields. Maybe notice these joins are slow and make them optional; build some expanded=true or with_subobject=true type query params.
2b) for writes, notice that CRUD is getting hard to maintain as frontend can create n^2 possible states for n fields, and backend validation needs to consider all other fields for each update. Move complex state transitions to mutation/action/verb endpoints instead of PUT(any fields). Add your first “status” fields.
This gets you quite far, likely to be fine for 2 years and beyond. After this there are lots of options depending on needs, and overhead associated with more layers/abstraction;
3) use something structured like GraphQL, or just refactor the API to do away with directly updating objects. Maybe lean into Finite State Machines if you have a small number of objects with complex transitions, endpoints trigger those parameterized transitions.
Another option to consider is a server-side rendered framework with client-side hooks like Django+HTMX, Rails Hotwire, Elixir Livewire, where the frontend devs are working on styling, design, templates, and most/all the actual app logic is backend code. This gets you surprisingly far these days. If you are the one defining all the features, you’ll iterate quicker in this mode.
Thanks for the thorough comment, I appreciate your insights. I was thinking about something like htmx since it seems to be getting a lot of positive buzz lately. The only thing keeping me from going that way is that I can offload most of the frontend work to folks who understand it better.
For 2b, could you elaborate a bit? Is this like having dedicated handling for specific functionality? If I understand you, for example instead of PUT-ing some status on /myobject/{id}/, I could have something like /myobject/{id}/update_status/. And the benefit here is assuming that setting status is not a straightforward transition (maybe requires other transformations, etc), that this lets you better compartmentalize this complex logic instead of having one giga-endpoint for all PUTs?
RE delegating frontend work, it’s a valid concern. Might be worth it to see how much of the Django templating work you could have the FE developers do; you can often still split it so that they do the presentation work. (Great thing about Django/HTMX is you still work with HTML generated in template fragments). The “API” is instead the state that you push into the template context. It’s a really productive development loop.
On 2b, yep exactly as you say. Instead of updating arbitrary fields with PUT/PATCH, you would have an endpoint per action. (Or a mix of both approaches if it is more expedient).
This doesn’t make much sense for “data bag” objects with lots of potential fields to edit like a user’s profile. But say your object has a mutation that always takes one parameter to update a model with many other incidental fields, you can map that to a POST /obj/<id>/update_foo {param} which is basically an RPC mapping to a single function.
This makes most sense when you have objects that look like FSMs since in that case you might have a few transitions, each taking different args, and each dependent on a specific state as precondition. But I have used this for e.g.“users/<id>/activate” which updates the status, sets a updated_at timestamp, and cannot be undone. The alternative is something like PATCH the is_active field, and having some gnarly logic to handle bookkeeping around time stamps and preventing reverting.
Another way of thinking about it is if any combination of values is valid, it doesn’t help. But as soon as you need to handle state transitions and particularly when you have multiple, then simply updating raw fields results in extremely messy validation logic within your update endpoint.
For reads and offloading front-end work, I think the article's approach should be good for your use case. Just make sure that your backend code is cleanly written. Create proper domain objects to represent pages that you will be serving as json. Pass all dependencies into these objects to construct them.
For writes, a typical rest approach as described in the grandparent is good.
Using specific actions such as `update_status` vs. allowing any fields to be supplied is… I think both can work. But it's not easy to cleanly organize the latter. It might be important to split different groups of related fields into separate logical paths, as though they were indeed supplied via separate endpoints. In Ruby/Rails I had success doing this via modules included into the model. The advantage of letting any fields be updated in any combination is that this would allow you to build different interfaces (i.e. admin UI, user UI, using REPL console if your language supports it) with different combinations of fields, and expect them all to work. You don't have to do that if you don't anticipate this need yet, but it's also true that splitting this logic into different urls is not all that different from splitting its handling under the hood, while allowing all fields submitted at once.
Given that you're the sole backend developer with a focused vision for the app, the Server-Informed UI approach described in the article could work well for you. It simplifies the backend architecture and makes it easier to coordinate with your frontend team.
However, for long-term scalability, you might run into issues with this approach. If so, take a look at "Bounded Contexts," a technique in Domain-Driven Design that helps break down your application into manageable, scalable units.
Gotta say I disagree with this idea. RESTful APIs have become the norm for a reason, and in my opinion it's dangerous and damaging to suggest abandoning them for a whole swathe of back-ends.
Re: YAGNI. I worked on a product where, about 10 years ago, they decided to build proper APIs (for the first time), mainly to power their own front-end, but they also told the paying clients that they were welcome to use them. Didn't take some clients long to start using and loving those APIs. (Granted, they were enterprise clients, with their own significant dev resources, that's not the case for all software.)
Re: front-end doesn't really have freedom. Yes it does! If one day page A needs a list of X, and there's already a nice generic API endpoint to list X, then only a front-end code change is needed.
No, what we have is everything trending towards a monoculture where every solution is a minor iteration or flavour change to systems that solved problems for companies at scale such as meta/google.
We regularly see developers talking about optimising js loops, dom updates etc in products that are little more than blogs with a few interactive elements.
Most of us work on things that will get ripped out and redone within a couple of years, yet dev culture tells us we should be focusing on developing for every possible future with reuse and scale in mind, and that's unfortunately false.
> We regularly see developers talking about optimising js loops, dom updates etc in products that are little more than blogs with a few interactive elements.
In which I see the opposite. Developers don't optimize because "DevEx" is a feature of a codebase, but performance is treated as an afterthought.
> [pointless changes] in products that are little more than blogs with a few interactive elements.
I wonder if a lot of this isn't busy work to justify having engineers on staff.
You need engineers on staff to make changes when that is necessary, but you don't typically need to constantly be make such changes. At the same time the engineers can't be seen doing nothing, so this type of work is invented.
You are saying engineers should build software, that doesn't frequently require changes to fit some new use case? Outrageous. Then we would have to do proper engineering and make things flexible, anticipating potential use-cases. We would have to heed advice written in books such as PAIP (or more recently "Software Design for Flexibility"). Can we be expected to know literature of our own field?
There's a well-established answer to this that I've done for years that satisfies both and it's the BFF pattern. The crux of which is an orchestration tier. A simple pass-through API gateway which can interface with all your thoughtfully-designed, single-purpose APIs, extract and transform one or more payloads and serve up a single payload that maps one-to-one with a frontend. I've deployed this architecture for years on some large-scale sites. It works very nicely. Bonus is that you can manipulate and even swap out entire backend pieces without worrying about your frontend.
qlio [1] was a really elegant implementation of this pattern before it was named and popularized. That provided a nice decoupling of the FE/BE teams by allowing very quick iterations, with an early means to compose apis to simplify the marshalling.
I agree that pretending you have multiple clients sets up a more difficult bar to meet. I agree that bar might be overkill for some projects.
The idea that it is overkill for all projects is a leap I can't follow. Optimizing for greenfield development speed is something inexperienced devs often do, and that's what this feels like.
What would you say those reasons are though? I think they mostly boil down to cost.
I agree with the article, but mostly for a reason the article didn't press on: performance and responsiveness. Making loads of small requests sucks, and on a mobile, it sucks times ten. HTTP/2 and whatnot have improved things but when you have even a small dependency tree of data, you can't avoid the network latency and it can quickly add up to a second or more of unresponsiveness.
Obviously this doesn't apply (or matter) in some circumstances, but in many applications I think it's worth the extra cost.
This article is good advice to get a working prototype or a decent solo-dev MVP website that will be re-written later. I've spent the last two months cleaning up the work of a developer who thought this way. Here's what you run into:
* Front end and back end become conflated. Often times code that should run on the backend runs on the front end and vice versa. Often shortcuts like saving ui state using the same api endpoint you use to save data become complex, and slow down improvements to both the ui and the public API. Likewise data manipulation and in some cases even validation gets done on the wrong end, too.
* You confuse UI code with application functionality. Code used to save the order of columns, stuff you entered the last time a dialog was displayed, and the convenient data structures (i.e. easy, not best) for the front end leak into the back end. So you end up spending a lot of time building back end functionality that has literally no utility to anyone else other than the front end developer.
* You lose focus on your other users: developers who integrate and build on your product. Eventually, those developers become in-house people who are doing integrations and implementations for customers, and a messy API will really slow them down.
Better advice: separate the concerns and have a separate API UI backend. Build exactly what you need to deal with UI state, and component specific UI settings. Your front end team is a perfect proxy for a third party developer for your public API. Take advantage of that. If they can't use it, or they get paper cuts, then paying customers will have problems, too. You'll also be able to on-board developers faster (easier training) and have better documentation.
I have some experience with it both as a user and a developer.
Code used to save the order of columns, stuff you entered the last time a dialog was displayed, and the convenient data structures (i.e. easy, not best) for the front end leak into the back end
As a user, I want the column order to persist when I’m moving between PCs or browsers or private windows.
You lose focus on your other users: developers who integrate and build on your product
Having done enough integrations, I prefer “what you see is how you work with it” mode. Because requirements usually come in a form of “there’s a button on page X, it produces a table, take it”. But when you approach it from the API side, this user story appears smeared across complex frontend code with numerous API calls and you basically have to reverse-engineer this part. Includes hours of comparing sparse docs against the network activity tab in the inspector pane. Very tempting to just $.click() the button and rip these <tr>’s out of it.
That said, I don’t think you don’t need APIs. But your either-or dichotomy is wrong. If you need an API, then build it. But it has nothing to do with how pages work. Because if pages do something clever, and there’s value in it, your API users will have to repeat it. Why not push that cleverness back to where it belongs.
And finally,
Front end and back end become conflated
It’s a whole system. Drawing an arbitrary line between graphics and logic not where the boundary between them is and repeating it in company’s structure – that creates a point of view where they conflate. If you don’t do that, there’s no conflation. Everyone just does the whole task like “add an input there from this db column”, without having to start complex interactions or negotiations. This separation is synthetic.
For me back end and front end being conflated is a good thing. Because most of the time it ends up one application anyway.
If you need generic API you can make separate application that will be API.
So if you make domain driven design your front end is separate concern. API for third parties is separate concern.
Still in database you might have stuff not used by external parties but then you don’t expose it in your external API. But you can make models on your front end API.
Making “generic” solution might seem like saving time and money but as times go by and it turns out you have to fight with generic API to get things done because one is stuck in his ways - it definitely pays off.
Hard agree with the YAGNI conclusion. There's a certain minimal amount of complexity needed to build your system, you need a good reason to build out another layer of indirection / abstraction here, something like needing web + mobile for the MVP or offering users direct access to the API as part of the MVP. Remember: if you think there's value in a REST-style layer, you don't need to build a separate API to support that. You can just write a getUsers() method on the server instead.
What do you get when you throw out the API layer? Easier refactoring (no wondering who all the consumers of GET /users are). No implicit contract with users who have reverse-engineered your API (and complain if you break it). Easier security (who needs API tokens?). No organizational footgun that leads to a frontend/backend team split and then requires a group manager to get new features shipped, because of course your API doesn't actually expose feature-specific data for features that Product hasn't dreamed up yet.
Sure, if you're FAANG scale, it'll be easier to work with APIs. You'll have some Staff+ Engineer architecting everything and coordinating between Product and a half-dozen teams that'll take months if not years to ship, and everyone will be happy with that. Strong fences make good neighbors, and contracts keep teams independent of each other. But in the early days? Feel free to ship things that don't scale.
I beat the drum of complexity vs complication. Solutions to a given problem have an inherent complexity. That’s the minimum amount of sophistication and cleverness required to implement them. And then there’s complications (think of the watchmaker’s term) which extra features that don’t directly solve the problem but, well, they look kinda neat!
Strive to find that inherent, irreducible complexity and solve at that level. Resist the urge to say “for just 20% more effort…”
I think your comment doesn't really make sense. Unless you're somehow assuming that the one and only way to do web apps is to go the dynamic html path, I don't see how your suggestion can even apply. Once you have a frontend that needs to retrieve data from a backend, how do you even address that?
Your comment reads as if someone complains people don't actually need cars because they can just show up somewhere.
If you use an API, but in a way the author suggests (conformed to the UI, rather than an abstraction), what's the simpler/better way to do auth than API tokens?
I have depth of experience in a niche industry and I'm working to build solutions for that industry, but I'm still early on the learning curve of being an effective full stack developer to make those solutions.
> Easier refactoring (no wondering who all the consumers of GET /users are)
Unless your API calls are properly abstracted into function calls. No reason you can't just have a `getUsers()` function on the frontend, and then use your favorite “find all references” tool from there.
My personal take: Don't put it all in the same box.
Just have a api/v1/rest/ base path for modular well defined resources that can be used across your application, without breaking changes, or even shared to third parties.
Then have a api/v1/features for coupled endpoints that return non standard objects, and with a more flexible lifecycle.
Chances are you're better off having both instead of trying to cram all the use cases into one approach
Most times you won't need a well-formed (REST) API to start with, and many times you will never need it. Most small projects should start without one and create one when need arises.
Sure, you won't "need" them. But it's very likely that you'll have resources that behave very well for rest like operations - user comes to mind, for things like user profile page, user settings, user detail, etc
A rest endpoint is likely handy to use across your application without having to replicate the same serialization over and over again.
Also many frameworks remove all the boilerplate for those operations (eg. Django class views), so creating one is a really good starting point.
The problem is not building a general purpose API, the problem is over thinking it. There’s a sweet spot where you build the API just slightly more flexible than your FE needs at the time, yet without wasting time on making it fit ‘all possible future use cases’.
Then you can later on whip up a mobile app if needed, let the clients use it, whatever. And evolve it accordingly.
I disagree. Trying to generalize a single implementation into an interface is always problematic. And that's exactly what inserting an API between the FE and BE is if your only UI is a web page.
If, later on, you want a mobile app, then an API can easily be deduced from your current, working UI, instead of trying to imagine what the UIs of your web FE and mobile app have in common.
This is how you lock yourself into a single front end design. And when you get ready for the next generation of your app or site, now you’re locked in and changing that front end design becomes exponentially harder.
In case of a complete redesign, yes you're right. With a generic API you will have about as much trouble writing the redesign as you did the original design.
In practice, a complete redesign is an incredibly rare event, and a good problem to have. Usually design of individual pages and layouts undergoes gradual, incremental changes, driven by real needs.
A redesign that requires front-end restructuring to such an extent that the data supplied is no longer sufficient should probably involve backend collaboration. Otherwise, you're tasking your backend team with building a generic site builder, which is much harder than a specific backend for your frontend.
The complaint about general-purpose APIs is valid. I've seen teammates, including senior ones, jump onto the bandwagon of some monstrous "data service" at work where the entire API to the frontend was an ORM. Absolute trainwreck.
This article advocates for another extreme that I've also seen at work. They invented some UI language in JSON and built some default JS components to render it. It works for very simple cases, but it's too rigid for anything slightly advanced, so the end result is some very clunky UIs and unhappy devs. You said your experience was bored frontend devs; that's usually a sign that their value is going to waste, and probably not because the backend devs can do their job better.
There's a good middle ground that's pretty common. You don't define the API to end all APIs, you don't pretend FE and BE are totally decoupled, you just have the frontend team ask for what they need and let the backend team serve it. Maybe you get an endpoint per page this way, but it's just sending back the data to populate the page rather than the actual layout. Maybe a few things are separate endpoints, but a reasonably low number.
> This article advocates for another extreme that I've also seen at work. They invented some UI language in JSON
This is called server-driven UI, and not what the article is advocating. I try to make that distinction by calling the whole practice server-informed UI, rather than server-driven UI. The article is meant to be advocating almost exactly what you wrote in the 3rd paragraph.
Your note says, "A Server Driven UI is when the API tells the client which components to display and with which content." Your description of server-informed UI is, "Render concrete boxes, sections, paragraphs, lists. Render the visual page structure." What's the difference?
I was thinking these keys are special and a different page would also have "section1" with "topBoxTitle," but maybe I misunderstood. If these keys aren't special and the frontend dev is just humanly interpreting this differently on each page then defining the actual layout to display it, that's probably fine. But I wouldn't describe this as rendering the visual page structure in the backend.
It may totally be my bad for not making this part clearer in the article, but these structures are never meant to be fed into something that can automatically produce a UI. They are not meant to be consistent.
The way I've been describing this lately — the json structure should come from you squinting at the page and seeing approximately what needs to be on it, and creating as-flat-as-possible way of sending it over, that makes it clear enough what each field is for, but not so granular as to be automatable.
If you have a designer in your company, I've been increasingly preferring to use their terminology to name fields. If page has "cards", "navigations", "banners" whatever designer calls them, call them the same way in JSON. This is a good middle ground. It makes sure that back-end passes what front-end needs, while at the same time making sure that front-end doesn't get the wrong idea that their framework can dictate and accept this json structure directly.
So if a full JSON-based language would allow you to customize every page element in any way you like, this approach only lets you to send over decisions about this page, using a json payload specifically for this page and the ways it can change, no more no less. You cannot add an arbitrary box or an arbitrary section in json, and expect it to appear.
I'd caveat that it can be a good idea to make it a general api if you're building a B2B app. These days, purchasing checklists often include "do you have a public api?" Saying no to that can be a sign of product immaturity that is worth trying to get ahead of, and having to maintain internal and external apis is a lot of overhead.
It also can be good as you scale to have an API interface you can have a services team learn how to work with to enable "special" flows for customers.
I don't know when we started splitting people into "frontend" and "backend" teams (FAANG influence?) and for what reasons, but 10 years ago we hired people with a general background, smart and curious enough to learn SQL, HTML/CSS/JS and whatever language you want to use on the server, able to build some end-to-end CRUD features on their own. And that's like what 99% of the companies need.
My point being: when you have the same person building the UI and the API, you avoid a lot of the problems described in this article.
After a couple decades of doing this… I’d pin it on an explosion of complexity and scope in the frontend.
It used to be if you could find a solid engineer with some vague aesthetic sense they could efficiently carry something through from conception and design to completion.
These days the scope of frontend development has increased to the point where learning and staying on top of it has become too large (in my opinion) for most people to do while also becoming and staying highly competent in everything else.
A “full stack” developer these days is supposed to understand frontend development, backend development, systems administration, cloud operations, and what else? All of these are massive areas that are constantly shifting.
By way of analogy, we used to have the town barber handle haircuts and shaves but also surgery because the main skill to master was precision with a blade. Then as the scope of being a surgeon expanded to include things besides being able to cut precisely, that became its own specialization. Further on, even within surgery each area of the body started to require so much specific knowledge that we further broke that down into all the different surgical specialties (heart surgeon, podiatric surgeon, brain surgeon, etc).
> These days the scope of frontend development has increased to the point where learning and staying on top of it has become too large (in my opinion) for most people to do while also becoming and staying highly competent in everything else.
I feel like frontend complexity is just a meme on HN to be honest. How complicated is it to learn a framework like React (which celebrated its 10 years 3 months ago) if you have a background in software engineering and some basic knowledge of HTML/JS? Like an afternoon to read the doc? 3 days to practice and be totally fluent? I managed graphic designers who knew nothing about computers but were able to learn React and be very productive. Then you pick one tool to build your production app. Yes there are many options, but you choose one (I use Vite.js) and stick with it for the next 5 or 10 years.
> A “full stack” developer these days is supposed to understand frontend development, backend development, systems administration, cloud operations, and what else? All of these are massive areas that are constantly shifting.
There is a new big thing every 10 years or so. Like the last one was Kubernetes and it almost 10 years old and you probably don't even need it. HTML5, ES6, React, Python, Django, SQL, Postgres, Docker, AWS, Terraform... all those tools have been there since almost forever (it feels like it) and mostly never changed. And you can build 99% of the web with that stuff.
As I said in another comment, yes it's a lot if you are a junior dev fresh out of college but you probably have some solid knowledge of all of that if you have been doing this for a very long time. At least enough understanding and experience to know what to google when you need to do something and what to do with what you found.
It's not that tough to keep up with FE. React has been dominant for closing in on 10 years now. ~2015 was probably peak framework fatigue. If you left the industry in 2016 and came back now, you could probably get productive and you would even recognize a lot of the tools.
The HN meme is that "Well ackshually you could build Figma with ASP.NET Forms, just like we used to do back in my day." That just doesn't resonate with anyone doing the work day to day because applications themselves (not necessarily the front-end tech) have a much higher bar today.
How complicated is it to learn a framework like React (which celebrated its 10 years 3 months ago) if you have a background in software engineering and some basic knowledge of HTML/JS? Like an afternoon to read the doc? 3 days to practice and be totally fluent? I managed graphic designers who knew nothing about computers but were able to learn React and be very productive.
The idea that React 10 years ago is the same as React today seems to be highly oversimplified. React was simple, then hooks were added, and state management fads that turned something simple and intuitive into an IQ signalling exercise. At least that was the case when I switched to back-end only dev a couple of years ago. And of course other parts of the stack also keep evolving: TypeScript (which is probably a positive, but again, I know people who do bizarre things with the language to build their "nerd cred").
Off the top of my head there was also Grunt/Gulp/Webpack/whatever is around now. NPM-Yarn->NPM. Angularjs->Angular 2.0+.
How many different CSS positioning methods are there? (that's what led me to leaving front-end, I was expected to do a pixel-perfect rendering of a design for a time-critical back-office app for some reason).
I would argue that it’s never been easier to be a full stack developer. You can manage all this complexity with one programming language, Typescript, with some HTML/CSS sprinkled in. You can literally define your infrastructure using AWS CDK in Typescript, write your backend services as lambdas, write your frontend using React, and you’re good. Compare that to 20 years ago where you were manually managing a server (probably physically setting it up as well). Maintaining a bunch of magic bash scripts to handle spinning up different services. Maintaining a backend written in a different language then your frontend. Managing a bunch of complex ORM garbage as a result of using bespoke backend language and having to convert it into usable data for the frontend. Also still learning JS, HTML/CSS for frontend. And don’t forget some jquery if you need some interactivity without a full page reload.
I can literally spin up a globally replicated service with as much scalability as I need using 1 programming language within an hour today. I doubt that was possible 20 years ago.
We used to have to pay 5 developers to maintain all these moving pieces, but now it only takes 1 developer to maintain all this stuff because so much of it has been simplified.
For someone who has been in this field for 20 years I remember starting out doing every aspect of webdevelopment myself. Then the bigger and more complex the projects became, the less that made sense.
The requirements of the distinct fields have gotten to a point where it's very impractical and hardly worthwhile for one person to master them all. You can't expect someone to translate a design into modern components that support all types of devices, resolutions and contexts to also come up with optimized SQL queries and knowing how to choose between Docker and Ansible for their distributed serverless functions that handle a variety of external endpoints.
Ofcourse not every project that uses this approach needs this approach, and there's still a market for the Do-It-All developer. My point is that the webdevelopment world of today isn't that of a decade ago.
> You can't expect someone to translate a design into modern components that support all types of devices, resolutions and contexts to also come up with optimized SQL queries and knowing how to choose between Docker and Ansible for their distributed serverless functions that handle a variety of external endpoints.
Why not? Obviously you can't expect this from someone fresh out of college/bootcamp, but that's what I do on a daily basis and I'd expect this from anyone with 10+ years of exp.
In a way, this article tries to bring the alignment between the frontend and backend closer to the full stack days, while also acknowledging the increased complexity of the front-end, that may be deserving of its own team focused on great UX.
It might be that for the author specific case (which we dont know in details) maybe this is a good approach.
In general I disagree with almost any idea that tries to fix some problems by adding coupling between components. I think the main thesis is where there is a false assumption:
> Your frontend can finally focus on presentation and UI. Your backend can finally focus on implementing exactly what’s needed.
If you make backend create one API call for each page, you will NOT get FE focusing on UI and backend on something else. You will get FE focusing on UI and BE focusing on UI and something else.
I do get the feeling of rebellion against common wisdom that is FE and BE as separate components.
When I was junior programmer (not suggesting here that author is junior, just talking about my experience) I often found myself rebelling against common programming wisdom.
Now, after more than a decade of seeing so many changes and request I think those old programmers that gave us that old wisdom about separating concerns and losse coupling are right.
I could boil down probably all old advice into the following 3 small pieces:
1. Write code. Not too much. Mostly simple, short and loosely coupled.
2. Go against common wisdom when you understand the tradeoffs. When not sure, pick the common wisdom.
3. You can maybe fix technical issues with more people, but can never fix organizational issues with code.
Author here. Here's an additional piece of wisdom that I picked up in over 20 years of doing this. Reliable communication allows for simpler, less coupled interfaces. Unreliable communication demands increasingly elaborate and coupled interfaces. I wrote a short note explaining this principle. https://notes.max.engineer/reliable-communication-allows-for...
> If you make backend create one API call for each page, you will NOT get FE focusing on UI and backend on something else. You will get FE focusing on UI and BE focusing on UI and something else.
You need your backend to focus on something, otherwise it would be focused on nothing/everything. You need to tie yourself down to use cases, otherwise every use case is your use case.
First let me again say sorry if my reply made you think I assume you are young. That was not my intention.
Second
> You need your backend to focus on something, otherwise it would be focused on nothing/everything. You need to tie yourself down to use cases, otherwise every use case is your use case.
What I was saying is that your backend will now do what was doing before AND also will need to handle UI elements and relation between them.
So you have duplicate data architecture/knowledge in UI:
- once in FE as it needs to know the data architecture
- again in BE where it also needs to know what data structure UI needs.
Thus it couples FE and BE together around key names, JSON structure and moreso on information architecture from UI concepts.
I am not saying you should not do that. Maybe it works for you. I would not do it.
For me it will be like saying in a traditional simple HTML app that I should name tables and columns in DB as they are in the UI. Maybe some might coincide but that will not be an architectural constraint when thinking the DB.
Have you looked at this from the angle of object constructor or function parameters? Let's say you build an object called Person. They have first_name, last_name, phone_number. That's what you probably will pass into a new Person. Now let's say you build an object called BusinessCard. Most people will be tempted to pass in BusinessCard.new(first_name: person.first_name, last_name: person.last_name, phone_number: person.phone_number) or simply BusinessCard.new(person). What I suggest is that you should pass in BusinessCard.new(main_text: person.first_name + ' ' + person.last_name, secondary_text: person.phone_number). This, but for webpage. Your front-end defines the constructor parameters for the page, and back-end passes them in.
> your backend will now do what was doing before AND also will need to handle UI elements and relation between them.
So you have duplicate data architecture/knowledge in UI
I don't think it will need to handle any elements, it will only need to construct the page by passing in what the page needs, not what the backend has. Just like constructing any other object or calling any other function.
In a way yes, you can say that any caller that constructs any object couples itself to said object, because it passes the specific arguments needed for its construction. Instead, we could provide access to the entire database, and allow objects to construct themselves by fetching any data they like. I guess I can agree with your characterization, but this would be considered an unjustifiable anti-pattern in any other context, because it makes it easy to lose track of dependencies, and makes it hard to optimize things. "Let's have objects auto-construct themselves from global state."
> it will be like saying in a traditional simple HTML app that I should name tables and columns in DB as they are in the UI
This part seems incorrect. Just because you pass parameters to construct an object, doesn't make it like having said parameters live in a database in the same format as the object constructor that accepts them. The entire point is that they are not the same, and need to be adapted to construct various things (pages in the front-end among them).
Based on my personal experience general purpose APIs lead to backends for frontends, which is a colossal footgun unless orchestrated by a mastermind. Even if you did have that kind of mastermind, if that person decides to leave you're left with a very expensive puzzle for everyone else to solve.
Here's my take: don't build general purpose APIs, build general purpose domain actions. A user making an account is a domain action. Sending a notification to the user is a domain action. Make doing each separately easy. Make doing both at the same time easy. Do this with all of your domain actions and now you have an easy way of building exactly the kind of endpoints all your front ends need.
Curious about this take, since I've done it successfully many times. Orchestrating a backend for frontend is usually a pretty trivial task. It's usually just mapping one json to another.
Here's just some of the things that become non-trivial in such a scenario:
* Validation - you either replicate the same rules on the generalized API/BFF/FE and risk inconsistencies or figure out a way to extract that validation to a shared package
* Making API calls inside the BFF - usually generalized APIs have a lot of features in individual endpoints, some of which aren't always documented so you'd need a really good documentation that gets constantly updated or some kind of a connector package that makes it easy to discover those features
* Testing - most of your endpoints are just calls to another API, all you can do is mock those calls in your tests based on what you expect the API to return but this gets hairy if you're just copy pasting the same 200 line JSON body across multiple tests so you'll need some helpers/a package for that too. It also requires that you always have a person on your team who's very familiar with how the API you're making calls to works so you don't make false assumptions
You either need someone who's aware of every little detail that goes on in both the generalized API and the BFF or you need all of these constant abstractions to be able to scale properly in any shape or form without introducing bugs with every new feature.
As a user, this approach often leaves things in a difficult state. The front end works, but automating data access is a hard slog through inconsistent API that's not fully revealed. Reporting is dependent on what is planned for the front end, not what is needed by the user. Because the front and back are tightly coupled, everything is fragile unless it's baked in.
I think many business apps should focus on having an API be a first-class citizen, and there needs to be better standardization in how they're organized and used. GraphQL could be part of that solution. It's also nice when the API lets you do the things you can do from the front end as well as more data-oriented tasks. For example, in Zoho Books, reporting isn't automated by the API, you'd have to roll your own or script the UI. The API should cover both reading and writing data items like transactions and downloading a PDF or Excel report using the reporting interface.
My experience working on "medium sized" B2B apps in 10-20 people teams is that the cleanest way to design API-s is to have a regular REST API for the basic CRUD operations on the entities, and then have more specialized endpoints for specific pages, tables, dashboards etc. on top of that. Having specialized endpoints is kind of required, as the data often needs to be searchable, sortable, filterable and paginated, which would be pretty wasteful, or sometimes even impossible to do on the frontend. I find these kind of REST API-s pretty straight forward to design and implement for the most part, and as others pointed out, onboarding is extremely easy if the new developer on the team has worked in software dev at least for a year or two. This also makes it pretty easy and quick to transition to a fully public API if needed (although I have rarely seen this as a requirement).
I totally agree with the general idea, and this is generally how I build backends these days. If there are multiple clients, it's usually even ok for each to have a backend which talks to the "master" backend.
But I will never get on board with sending things like content or UI related stuff from the backend. I've seen too many times, engineers think "all our forms are the same, let's just have the backend send a list of input field names and types like 'address'". It's almost always a bad abstraction, causes pain on the frontend, and restricts the creativity of engineers and designers.
The article doesn't disagree with your second paragraph. It encourages to treat each "page" as a unique snowflake, including forms in it. I should probably do a better job to discourage consistency and standardization of these json structures. This is addressed in another one of my comments if you search for "my bad".
I read some of the HTMX essays [0] last week due to it being accepted into the GitHub accelerator program.
So, a fully RESTful API must output HTML. I'm not going to go into why but that's what Roy Fielding (the creator of REST), Martin Fowler and the HTMX guy(s) say.
The HTMX guy(s) also say that of course this fully RESTful API that outputs HTML is sometimes bad because you don't want to have to parse data out of HTML. So, the optimal setup is to have both a fully RESTful HTML API and the more common JSON API.
I also think that for the fully RESTful HTML API, it doesn't make sense for the outputted HTML to have any JS.
So, the question I'm pondering at the moment is, is a fully RESTful HTML API simply a website that doesn't use any JS? If so, then the optimal setup is to have an SSR website that doesn't use JS and a JSON API. Of course, under the hood, the website and the JSON API use the same lower level API (eg. SQL queries and back-end functions).
What's crazy for me personally is that this is exactly how I have set up Comment Castles. There's the no JS website [1] and the JSON API [2]. But I didn't build a no JS website because of REST, I just prefer websites that have no front-end JS.
> is a fully RESTful HTML API simply a website that doesn't use any JS?
No, I don't think so. Sprinkling a little client side JS can certainly be beneficial and doesn't invalidate it being a RESTful HTML API. _hyperscript [1] was created by the htmx author for that purpose. There's a book [2] that you may not have seen written by some core htmx dev's that goes into this [3] in more detail.
I have building most of my modern sites with NextJS and could not have been happier as it gives me the flexibility of directly quering DB, I dont have to worry about the general purpose API. But if I wanted to I can just easily build it too.
Don't think I could disagree any harder. Requirements change. Business needs change. You want your FE to be able to evolve independently without having to wait for the BE to implement the precise logic you need.
As well as your FE team forced to wait on BE changes for new features, you've also created busywork for the BE team anytime there's a FE refactor/restructure to boot.
An organisation that separates frontend and backend teams will naturally converge to an architecture where the collaboration between frontend and backend is minimized (i.e., a generic REST API) -- Conway's law.
This doesn't mean that an actual cross-functional team that contains both frontend and backend developers can't use the proposed single-purpose API to run circles around the split team.
I see the point of the article, and even on a principled level I might agree with it (e.g., don't overly abstract). A few problems I see though:
1. If you make backend-only abstractions to keep your code even a little bit DRY, you'll have a duplication of code (both the abstraction and the BFF endpoint). I think it'd be better to go all in on the abstraction up-front.
2. This will impair the conceptual integrity of the system and arguably lead to development slowdowns in the long run. To quote Mythical Man Month: “I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas.”
3. Your ad-hoc logic will likely be duplicated anyways across pages.
4. A general purpose API makes security features like RBAC much simpler.
I think a general-purpose API is a well established abstraction you can rely on for pretty much any project.
This is good general advice. Sometimes you are building a general library or API, and sometimes there just happens to be a network partition between two components that make one module. My general rule of thumb is that it takes 3-5 times as long to build a library/generic API over regular code. It also requires a very different mindset.
Should you do that sometimes? Absolutely! In big companies you probably have entire teams doing this for API’s that are entirely internal. Just recognise that it is a huge cost and only do it when the benefits are likely to outweigh the costs.
That’s not to say that you should never think about generalisation when writing normal code, (one of my pet peeves is interfaces taking a single “something” when a collection could just as easily have been used), there are many ways code can evolve and some up-front thought can save a good deal of refactoring later. Just don’t write a generic library when domain specific code will do.
This is BFF pattern described many years ago. Here's the standard way I build this kind of thing: Develop your APIs per functional or really per system. One for content, one for identity, one for ads or whatever. Then you build an orchestration tier that does pretty much what OP says. It stitches the various services together and returns a page model. The ideal is to reduce round-trips and cognitive load on FE dev. If you need the top 3 pieces of relevant content, plus an ad plus the user's first name, return all of that and nothing else. The last piece is just SSR. Use next or nuxt or whatever and you can output HTML on the server or stich it on the client with one codebase seamlessly.
I disagree. I understand a lot of people enjoy the simplicity of HN, but when you find a developer who really knows how to take advantage of the additional "bells and whistles" that come with modern browsers, without making them annoying, the experience can be significantly better than HN.
Counterpoint: I did this and it worked out great. We ended up wanting to support direct API access from our customers and we were already in a good place to do so.
YMMV. An observation I have is that really flexible APIs make it easy for the frontend to make small changes, instead of needing to coordinate with backend changes. On the other hand this can lead to maintenance problems down the road as it’s hard to control exactly what data/state your system can transition to.
I view this as one of the best pieces of tech debt to take on in an early-stage startup, basically keeping your options open and minimizing short-term friction, at the expense of taking on a bit more cleanup after you find PMF and start to make your backend more robust for the usecase you are scaling.
this is fine but makes the backend extremely slow and hard to maintain. it's effectively just glorified RPC
- restful api low logic on backend and lots of logic on front-end
- grpc style as the author suggests, more logic on backend no logic on front-end
- keep api restful on backend, add backend for front-end(graphql/rpc), front-end remains simple, most logic is in the middle. this also allows you to isolate presentation spaghetti code in one place, keeps your backend api clean for use for other clients
worked in all 3 probably having a custom presentation layer works best but require more setup upfront. another advantage of having a restful api and have the front-end do more work is being able to incrementally load data into the UI therefore reducing TTI/TTFB
Don't build a general purpose API so your front end can have half the the logic outside.
Just build it the way you would as if it were local, and then hook it up to a back end with a git-like synchronization mechanism. You know, the same way you do all your own development.
If you don't know how to do that, maybe you could learn that instead of arguing over REST vs GraphQL, or whether your line format should care what boxes are on the page.
All I know is, it's fantastic to be able to add features to a front end, persisted in a back end, without having to write code on both sides.
"What about validation??"
Guess what, if users each have their own workspace, isolated and data driven, you don't need to worry about one bad request ruining your whole service.
General purpose API helped us when we had several features to implement in a row on a tight schedule: when I was busy implementing feature N+1, our frontend dev was still polishing feature N (with changing UI/UX requirements), and she didn't need me (the backender) because the backend just exposed the model as is without knowing what the frontend looked like. She could make lots of changes to the UI without needing any changes to the backend. Yeah it translated to more API calls but it allowed us to decouple our work completely and deliver more features in parallel.
Reading this may give the impression that non-api-decoupled things like e.g. ls or curl or inkscape cannot be developed in parallel.
Also, think of it this way: if there was no general purpose API, it could be (figuratively) May instead of August when you did N and N+1. I mean, there’s no guarantee that it would, but this potential benefit is relative to a different baseline than in TFA.
It sounds almost like the author is advocating for a really dumbed down BFF layer (backend for frontend) but what they are missing is behind the bff layer you would have general purpose apis. And if I interpret the article to the extreme what is the difference between this pattern and how html is currently served. Salesforce works sort of similar to what is described and it’s a nightmare, these restrictive ideals in my experience just aren’t good in practice because they tend to slow down development.
And the htmx-ification trend continues. I guess this is sort of the counter movement now to all the API-First/Data-First/SPA-First evangelism of the last few decades.
I don't quite understand why though. Or rather, why those posts always seem to have an intense emotional dislike to anything data-driven.
What exactly is wrong with just using React? Or, heaven forbid, plain Javascript, which browsers have spent the last decades optimizing it make working with REST APIs and JSON as easy as possible.
Can you expand on "just" using React? Are you saying "what is wrong with using React (the framework including React the library and all of its friends)?", or are you saying "what is wrong with just using React (the library) only?"
I was more thinking of React, the framework, with as little addons as possible. React's data model always seemed the most reasonable to me, if you have to use a framework or library at all.
I think the author is generally correct that all data should be provided in a single request. And to take it a step further, you should be able to change your accept header to JSON to serve an API. API/dumb-frontend aren't mutually exclusive.
In fact, this is what both GitLab and GitHub do. Try it out!
This is one thing I’m loving about Remix. It’s how the framework works, so there’s no need to enforce this pattern or educate folks about it.
AND Remix’s nested routing still has you break down the data loading into a couple different parts (1 per route segment).
So it feels like you’re not loading everything in one place, and may be less likely to make people feel like they’re doing something “different” or “wrong”. It’s
I am not a big fan of building extremely fine grained general purpose APIs for frontends. A lot of the burden of juggling with data and presenting it is shifted to the UI layer. I am also not a big fan of building a composite API for "pages". Now you have a tightly coupled frontend/backend communication. What happens when we split the page or have to change the format of the data? Also the response of such an API can be quite big and nested. Ends up being a dump of data with poor documentation.
I would try to find a middle path. Building composite APIs which have a concise purpose for the frontend but are modular enough to be reused across different areas of the application.
Also, I feel that most of the problems discussed in the post are amplified when we have different groups working on the frontend and backend. If a single team is responsible for both the backend and frontend, API decisions are localized and can be changed as the development progresses.
The past 2 years have seen some advancements in full-stack frameworks with streaming components. Such as NextJS with React Server Components or .NET Blazor.
Those would be my preference for an app that doesn't need a public API. If each partial layout and page component does it's own data loading server side, you can skip GraphQL or making a Full Page API and partial update APIs.
Part of the problem with this is organizational though. For the past decade teams have largely been split between frontend/backend who typically work in 2 completely different languages/frameworks. The steaming components approach is a switch back to full stack and a single language/framework. Also every language doesn't have one of these streaming component frameworks, so it's not really possible to do an app completely in Go for example.
> This is similar but not quite the same as Server Driven UI1. Perhaps we could call it Server Informed UI.
I think we should just move back to server side rendering, JSP and even ASP were great, plain HTML, some JavaScript where desired since we can't seem to get away from JS.
I read this and got to the end, and laughed because that's what we're working on (more a composition tool from different data sources using a headless CMS and a Knowledge Graph) - so in the end we do need a general purpose API since that's part of the USP.
So basically an old style MVC framework style app like RoR, CodeIgniter etc but made with a JSON response the describes the page rather like Shopify's Store Theme 2.0?
Sounds great. Now since you have access to the server code (unlike Shopify's system described above) just return HTML and use your JS flavour of choice in the places where you need heavy JS ... like we used to make web apps.
The happiest web devs I speak to are those working in the older MVC style, mainly RoR people. While everyone in AWS/Serverless/JS land is burning out and miserable with the house of cards stack we are balancing React on top of.
I've built both models (and more too) over decades.
There are lots of questions one might ask before deciding between these two (or other) approaches:
* Where does this team want the boundary between client and server in terms of user experience, server load and substantial UX interaction latency?
* Can this frontend team independently plumb a changes all the way through the database?
* Are backend teams always available for (or interested in) every minor update?
* Are both halves of the application deployed on the same schedule?
These are a handful of the first questions I'd ask before trying to push either solution at this point in time, the answers would likely lead to more questions.
Have to disagree with this. API is not always for FE. It can be consumed by other service, or a script. Making generic API helps reuse of common API across web pages. For example notification, user profile.
What’s missing from this article is a discussion about 1) team composition and software development lifecycle, and 2) which libraries are available to help easily provide and consume the obvious RESTful operations.
1. Team composition
If you have a separate backend developer from front-end developer (or worse, on a completely separate dev team), then making a separate endpoint for each page means you have to manage the feature dependency. As in, you need to make a ticket for the backend work first and the front end work can only start once the backend work is finished. And you need to do this for every single new page you make. On the other hand, if the backend team in advance just makes a generic rest API that provides all access to all of the obvious resources, it gives the front-end team the freedom to develop on their own timeline.
If all your teams devs are full-stack, this is simplified a bit but still you have the problem of deciding to intelligently architect your data access layer to not have massive amounts of duplicated code.
2. Supporting libraries
Often it’s possible on the backend to just easily make restful crud for every single resource that makes sense. A single backend dev can do this in one go without much conversation besides the initial one about access policies. Similarly, the are often front-end libraries that simplify not only writing getters for these endpoints but also for things like handling local caching, business object creation from JSON, and refetch logic.
Obviously the libraries that are available and how well they integrate with your workflow will depend on your front-end framework. React, for example, has libraries that directly connect your rest access into specific components.
—
A caveat: obviously the pro-con landscape changes if you have serverside operations that either:
A) feature some interaction that is sensitive to race conditions, B) requires a guarantee that one data type not get written unless the a previous write succeeded, or C) would require special queries for performance reasons
In these cases, I’d say make a custom endpoint to encapsulate the correct/safe behavior. But these one-offs can be created only when they are needed. No premature optimization/complexity is required.
> you need to make a ticket for the backend work first and the front end work can only start once the backend work is finished
In practice, a lot of this can be done in parallel. Once the design is established, front-end can start working on the layout/UX without knowing the exact shape of supplied data. The back-end can start proposing the shape of data and populating writing SQL/other queries to populate it. The final sync will only require small adjustments.
> it gives the front-end team the freedom to develop on their own timeline
At the cost of backend having to support every single use case under the sun.
> still you have the problem of deciding to intelligently architect your data access layer to not have massive amounts of duplicated code.
All this involves, is passing the pages the data they need for their construction. The alternative is building and supporting a generic data access for infinite use cases that is hard to optimize.
I see this suggested often, and I don't think it's true that the way to reduce duplication is by providing all data forever. You can provide specific entities wrapped in pages, and reuse the backend logic used for their construction.
> it’s possible on the backend to just easily make restful crud for every single resource
This shifts all the complexity of combining data onto the front-end, and the front-end is a lot more distant from the data storage. See the note where I describe that "Reliable communication allows for simpler interfaces"[1].
Better, just don’t write an HTTP API for your front end. Use a server side framework. Done.
It’s a pain coming into a project that was built the way the article recommends. Endpoints become RPC calls instead of resources serving entities. The hodgepodge of “API” calls becomes difficult to scale horizontally due to implied (and never documented) data dependencies. So you miss out on transport layer caching, entity caching, and a host of other goodies.
All because your front end “framework” wants to live in a separate codebase from the server.
The "page" becomes the resource for reads. You get great scalability, because you can use your database in the most efficient way possible to provide entire data for the page. You can maintain 100% back-end clarity on what each page depends on (instead of letting front-end request anything they want in any part of the view).
I personally prefer server-side frameworks. I also understand when a front-end team prefers to work a certain way, has a lot of knowledge and tooling built-up around a certain way, and is worth accommodating for that reason.
We used to call this common sense. It's insane we drifted so far from being sensible.
I genuinely think 90% of frontend developers should not exist: the web would be better and companies would have better products.
I include myself in the frontenders, I do terrible apps which should just be simple static pages with a sprinkle of js - in my defense, I need the money and nobody listens to me, they say this is modern frontend development.
For my tech not-savy clients I can deliver awesome and simple solutions without using the dreaded R framework.
Quite surprised at how much people agreed with this post. I personally don't think this is such a good idea - it's not that hard to write a general purpose API that covers 80% of obvious use cases, and have app-specific grouping of endpoints for the remainder. And I include things like rollilng multiple requests into one as part of the obvious use case. Because it's usually very easy to implement - either copy/paste, or 5 lines for more complex solutions like an orm, graphql, etc.
It seems like a terrible idea to couple the backend with the front-end so strongly.
Seems like an over correction. Yes trying to make your front end API be your public API is probably a bad idea, but making your front end API completely coupled to the structure of your app structure is probably a bad idea too. Best approach is probably in the middle somewhere.
It probably comes from misusing the API route and feeling they want to keep it simple, but instead heavily coupling things.
A motivated BE that wants to be productive avoiding unnecessary work would probably hate being "support for frontend" whenever they decide to change something and a motivated frontend would probably want to turn themselves to full stack powers to not depend on someone else.
It feels really bad as a pattern unless that webapp is your only concern and is constantly moving or something, or if there's no actually FE or BE but just few people maintaining stuff as a side concern (backend for frontend works there).
I'm finishing a project now, using exactly this design pattern. Sort of backed into it, motivated pretty much by the issues outlined. In addition, I wanted something that is clear and sustainable far into the future. The simplicity of this approach supports that. Quite liberating.
Complexity is the enemy.
True, one might reasonably propose a number of what-if objections. But for many (most?) projects), imo this is a good approach.
An alternative to what's suggested in this post, how about going API first or even API only? Don't build a frontend at all, just build an API. E.g. you can deploy things on fly.io just by using their GraphQL and REST APIs. The advantage of focussing on the apis is that other developers can much easier integrate with your service and embed it into their own applications.
I’m going to get zapped once again, but if you’re building complex applications, custom read models automatically updated based on events from operational writes is the optimal way to connect a front-end to APIs.
Each read model should have a distinct business purpose.
Generic models will only confuse maintainers and they’re very likely going to see such code as brittle and tech debt.
If you do proper hexagonal architecture it wouldn't actually matter and you can replace REST with graphql or feature-based endpoints. Before you rip that out, start with that. (YMMV and don't apply every piece of advice to every problem, that's the biggest problem)
It's also an ideal vector for scraping your site-specific data if it's an XHR request, typically an enumerating id or something a sitemap will spell out for more sparse IDs.
doesn't this require you to nail down design requirements?
If the designer gives a few different comps to try out, or wants to A/B test something, they'd have to align those changes with both backend AND frontend people to make sure they both know what the changes are.
If the data dictates exactly what the frontend should look like / placement of content, shouldn't the backend just send styling / html / app code directly, and just skip frontend completely?
When are we going to get away from all the YAGNI engineering that has sold short the promise of the Web for more rented silos, and get to something more like HATEOAS?
This only works for small projects.
Data-heavy frontend usually sends multiple requests to allow async rendering of different parts of a page with different loading times.
This also doesn't work if your design changes a lot. AKA a design change will now mean a backend re-design too.
Your first point is addressed in the article. For the second point, my experience is that the backend probably needs some work in that case anyway. If the backend is well designed, then only the presentation API endpoint needs to change. So a BE engineer is doing the tweaking instead of a FE engineer; not a big deal. Less of a big deal if there's no line drawn between FE and BE engineers.
Not so crazy. I can name at least 2 of the top 5 North American airlines running this in production for Mobile and Web supporting hundreds of millions of user requests, daily.
Why not just send HTML as a single response at this stage? Sometimes it feels like we are doing web development more complex than it needs to be.